Orientdb Java Heap Error

Question

I ran a test on Orientdb 2.2.10 using Transnational graph

With the following Sudo code on single thread system

   for(i = 1 to 1000)
{
    DB_Name = getRandomString()

    createGraphDb(DB_Name ) using OServerAdmin : if db do not exist
    gFactory = OrientGraphFactory(DB_Name ) : if db exist


    graph = OrientGraphFactory.getTx()
    previousEvent = null

    for(j=1 to 1000)
      {
        currentEvent = getrandomString()
        if(previousEvent and currentEvent != null)
              {
                   - create Vertex named currentVertex and edges between them 
                   - (from previous to current event)
              }
        else
              {
                    create vertex named current event 
              }
         previousEvent = currentEvent
      }
   graph.shutdown()

}

I got the the following Exception after few iteration of my outer loop.

Exception in thread "Thread-0" java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOfRange(Unknown Source)
at java.lang.String.<init>(Unknown Source)
at java.lang.String.substring(Unknown Source)
at java.lang.String.split(Unknown Source)
at java.lang.String.split(Unknown Source)
at com.orientechnologies.orient.core.config.OStorageConfiguration.fromStream(OStorageConfiguration.java:268)
at com.orientechnologies.orient.core.config.OStorageConfiguration.fromStream(OStorageConfiguration.java:497)
at com.orientechnologies.orient.core.config.OStorageConfiguration.load(OStorageConfiguration.java:207)
at com.orientechnologies.orient.client.remote.OStorageRemote.open(OStorageRemote.java:330)
at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.open(ODatabaseDocumentTx.java:257)
at com.orientechnologies.orient.core.db.OPartitionedDatabasePool$DatabaseDocumentTxPooled.internalOpen(OPartitionedDatabasePool.java:445)
at com.orientechnologies.orient.core.db.OPartitionedDatabasePool.openDatabase(OPartitionedDatabasePool.java:308)
at com.orientechnologies.orient.core.db.OPartitionedDatabasePool.acquire(OPartitionedDatabasePool.java:263)
at com.tinkerpop.blueprints.impls.orient.OrientBaseGraph.<init>(OrientBaseGraph.java:144)
at com.tinkerpop.blueprints.impls.orient.OrientTransactionalGraph.<init>(OrientTransactionalGraph.java:78)
at com.tinkerpop.blueprints.impls.orient.OrientGraph.<init>(OrientGraph.java:135)
at com.tinkerpop.blueprints.impls.orient.OrientGraphFactory.getTx(OrientGraphFactory.java:119)

Is their any problem with my sudo code or with OrientDb.

If their exist any better way, kindly please suggest.

Thanks..!

Answer 1

How many gigs of ram do you have available? For example if you can assign maximum 8GB to the Java process, it's usually better assigning small heap and large disk cache buffer (off-heap memory). So rather than:

java -Xmx8g ...

You could instead try this:

java -Xmx800m -Dstorage.diskCache.bufferSize=7200 ...

For further details you can read the page regarding performance tuning on official documentation.

Hope it helps.

Answer 2

You can check heap usage with -XX:+PrintGC -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintTenuringDistribution -Xloggc:$ORIENTDB_HOME/log/gc_%p_%t.log on server (can add latest XX flag only for investigation about promotion from new to old ).

You can add similar settings on your client (change gc.log option only). I've just run a java class (8u102 on Mac, with 16gb and i7), based on your pseudo code, but running 1 loop (1 db) with 1000 vertex+edges and it successfully completed the job without OutOfMemory

Bye Claudio

Answer 3

In OrientDB the transactions keep changes in RAM, so if you have transactions with hundred thousands of elements it could cause this problem. I suggest you commit every X elements. I suggest to start with 1000 and then tune it based on the result.