Neo4j server - slow import

Question

I have an application which used embedded neo4j earlier but now I migrated to neo4j server (using java rest binding). I need to import 4k nodes, around 40k properties and 30k relationships at a time. When I did import with embedded neo4j, it used to take 10-15 minutes, but it is taking more than 3 hours for neo4j server for the same data, which is unacceptable. How can I configure the server to import the data faster.

This is my what my neo4j.properties looks like

# Default values for the low-level graph engine
use_memory_mapped_buffers=true
neostore.nodestore.db.mapped_memory=200M
neostore.relationshipstore.db.mapped_memory=1G
neostore.propertystore.db.mapped_memory=500M
neostore.propertystore.db.strings.mapped_memory=500M
#neostore.propertystore.db.arrays.mapped_memory=130M

# Enable this to be able to upgrade a store from 1.4 -> 1.5 or 1.4 -> 1.6
#allow_store_upgrade=true

# Enable this to specify a parser other than the default one. 1.5, 1.6, 1.7 are available
#cypher_parser_version=1.6

# Keep logical logs, helps debugging but uses more disk space, enabled for
# legacy reasons To limit space needed to store historical logs use values such
# as: "7 days" or "100M size" instead of "true"
keep_logical_logs=true

# Autoindexing

# Enable auto-indexing for nodes, default is false
node_auto_indexing=true

# The node property keys to be auto-indexed, if enabled
node_keys_indexable=primaryKey

# Enable auto-indexing for relationships, default is false
relationship_auto_indexing=true

# The relationship property keys to be auto-indexed, if enabled
relationship_keys_indexable=XY

cache_type=weak

Michael Hunger Michael Hunger · Accepted Answer · 2014-03-23T10:43:59

Can you share the code that you use for importing the data?

The java-rest-binding is just a thin wrapper around the verbose REST API which is not intended for data import.

I recommend to use cypher queries in batches using parameters if you want to import more data. Check out RestCypherQueryEngine(restGraphDb.getRestAPI()) for that. And see restGraphDB.executeBatch() for executing multiple queries in a single request.

Just don't rely on the results of those queries to make decisions later in your import.

Or import the data embedded and then copy the directory over to the servers data/graph.db directory.

Neo4j server - slow import

1 Answers