0
votes

I have used the neo4j import tool,as follow: "neo4j-import --into ..\data\databases\graph.db --nodes "Node-header.csv,adm_Node.csv"

i am wondering why the heap size is increasing all the time.In order to importing 13G nodes and 10G relations data successfully,I have to using 64G RAM machine and set the Max heap size to 55G.

1

1 Answers

1
votes

You don't have to increase heap size as the import tool uses memory where it can, heap or off-heap, doesn't matter. What matters though is the number of nodes imported (not number of relationships) in correlation with how much RAM you have. There's caching done during import to remove random I/O and currently the entire node set of an import must be able to fit inside your RAM, otherwise the import fails. You should count roughly 20B / node needed, which would make your 13G node import impossible on your 64G RAM. Problems with big node imports like this is a known problem and will be fixed at some point (unfortunately don't know when at this point in time)