Apache-ignite: Persistent Storage

Question

My understanding for Ignite Persistent Storage is that the data is not only saved in memory, but also written to disk.

When the node is restarted, it should read the data from disk to memory.

So, I am using this example to test it out. But I update it a little bit because I don't want to use xml.

This is my slightly updated code.

public class PersistentIgniteExpr {
    /**
     * Organizations cache name.
     */
    private static final String ORG_CACHE = "CacheQueryExample_Organizations";

    /** */
    private static final boolean UPDATE = true;

    public void test(String nodeId) {
        // Apache Ignite node configuration.
        IgniteConfiguration cfg = new IgniteConfiguration();
        // Ignite persistence configuration.
        DataStorageConfiguration storageCfg = new DataStorageConfiguration();

        // Enabling the persistence.
        storageCfg.getDefaultDataRegionConfiguration().setPersistenceEnabled(true);

        // Applying settings.
        cfg.setDataStorageConfiguration(storageCfg);

        List<String> addresses = new ArrayList<>();
        addresses.add("127.0.0.1:47500..47502");
        TcpDiscoverySpi tcpDiscoverySpi = new TcpDiscoverySpi();
        tcpDiscoverySpi.setIpFinder(new TcpDiscoveryMulticastIpFinder().setAddresses(addresses));
        cfg.setDiscoverySpi(tcpDiscoverySpi);
        try (Ignite ignite = Ignition.getOrStart(cfg.setIgniteInstanceName(nodeId))) {
            // Activate the cluster. Required to do if the persistent store is enabled because you might need
            // to wait while all the nodes, that store a subset of data on disk, join the cluster.
            ignite.active(true);

            CacheConfiguration<Long, Organization> cacheCfg = new CacheConfiguration<>(ORG_CACHE);

            cacheCfg.setAtomicityMode(CacheAtomicityMode.TRANSACTIONAL);
            cacheCfg.setBackups(1);
            cacheCfg.setWriteSynchronizationMode(CacheWriteSynchronizationMode.FULL_SYNC);
            cacheCfg.setIndexedTypes(Long.class, Organization.class);

            IgniteCache<Long, Organization> cache = ignite.getOrCreateCache(cacheCfg);

            if (UPDATE) {
                System.out.println("Populating the cache...");

                try (IgniteDataStreamer<Long, Organization> streamer = ignite.dataStreamer(ORG_CACHE)) {
                    streamer.allowOverwrite(true);

                    for (long i = 0; i < 100_000; i++) {
                        streamer.addData(i, new Organization(i, "organization-" + i));

                        if (i > 0 && i % 10_000 == 0)
                            System.out.println("Done: " + i);
                    }
                }
            }

            // Run SQL without explicitly calling to loadCache().
            QueryCursor<List<?>> cur = cache.query(
                    new SqlFieldsQuery("select id, name from Organization where name like ?")
                            .setArgs("organization-54321"));

            System.out.println("SQL Result: " + cur.getAll());

            // Run get() without explicitly calling to loadCache().
            Organization org = cache.get(54321l);

            System.out.println("GET Result: " + org);
        }
    }
}

When I run the first time, it works as intended.

After running it one time, I am assuming that data is written to disk since the code is about persistent storage.

When I run the second time, I commented out this part.

if (UPDATE) {
    System.out.println("Populating the cache...");

    try (IgniteDataStreamer<Long, Organization> streamer = ignite.dataStreamer(ORG_CACHE)) {
        streamer.allowOverwrite(true);

        for (long i = 0; i < 100_000; i++) {
            streamer.addData(i, new Organization(i, "organization-" + i));

            if (i > 0 && i % 10_000 == 0)
                System.out.println("Done: " + i);
        }
    }
}

That is the part where data is written. When the sql query is executed, it is returning null. That means data is not written to disk?

Another question is I am not very clear about TcpDiscoverySpi. Can someone explain about it as well?

Thanks in advance.

Alexey Popov Alexey Popov · Accepted Answer · 2018-01-25T15:56:50

Do you have any exceptions at node startup?

Very probably, you don't have IGNITE_HOME env variable configured. And the Work Directory for persistence is chosen somehow differently each time you run a node.

You can either setup IGNITE_HOME env variable or add a code line to setup workDirectory explicitly: cfg.setWorkDirectory("C:\\workDirectory");

TcpDiscoverySpi provides a way to discover remote nodes in a grid, so the starting node can join a cluster. It is better to use TcpDiscoveryVmIpFinder if you know the list of IPs. TcpDiscoveryMulticastIpFinder broadcasts UDP messages to a network to discover other nodes. It does not require IPs list at all. Please see https://apacheignite.readme.io/docs/cluster-config for more details.

Apache-ignite: Persistent Storage

1 Answers