Overview for Persistence in Apache Ignite

Question

I think I don't fully understand yet the Apache Ignite cache persistence. I probably miss an overview.

What I would like to achieve is something like this: Three data nodes that persistently and replicated store the cache data either on their own separate disks or in single 3rd party DB. As long as one of these nodes is available, all data shall be available to the cluster nodes. Configs for these three nodes must have the PersistenceConfiguration, I guess? What about the backups setting? This must be set to 2? What is the correct setting that as long as one of the three node is available all data will be available?

Do all data nodes have to be available for write operations to the cache? Or is one enough and the other two will replicate once they connect?

Other worker nodes shall use the cache but not store on disk. Configs for these node shall not have the Persistent set, I guess?

Sorry for all these questions. You see I may need some background information for the data store.

Thanks for any help!

Denis Denis · Accepted Answer · 2017-10-11T08:13:48

Ignite native persistence can solve your problem. You can enable it by adding PersistentStoreConfiguration to IgniteConfiguration. Here is documentation on how to use it: https://apacheignite.readme.io/docs/distributed-persistent-store#section-usage

Every node that has persistence enabled will write its primary and backup partitions to disk, so when restarted, it will have this data available locally. If other nodes connect to the cluster after that, they will see the data, and it will be replicated to new nodes if needed.

Judging by your needs, you should use replicated cache. All data in cache will be stored on all nodes at the same time. When node with some data persisted on disk starts its work, it will have all data available, just like you need. Replicated cache is effectively equivalent to having all data backed up on every node, so you don't have to additionally configure backups. Here is documentation on cache modes: https://apacheignite.readme.io/docs/cache-modes

To restrict cache data to be stored on particular nodes only, you can create three server nodes, that will store data, and start other nodes as clients. You can find the difference here: https://apacheignite.readme.io/docs/clients-vs-servers

If you need more than three server nodes, then you can use cache node filter. It is a predicate, that specifies, what nodes should store data of some particular cache. Here is JavaDoc for CacheConfiguration.setNodeFilter method: https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/CacheConfiguration.html#setNodeFilter(org.apache.ignite.lang.IgnitePredicate)

Another option to enable persistence is to use CacheStore. It enables you to replicate your data in any external database, but it has lower performance and less features available, so I would recommend to go with the native one. Here is documentation on 3rd pary persistence: https://apacheignite.readme.io/v2.2/docs/3rd-party-store

Overview for Persistence in Apache Ignite

1 Answers