Inserting Data into Neo4j through neo4j rest binding Batch REST API becomes slow as more data is inserted

1
votes

I am currently trying to insert lots of data into neo4j. by using the neo4j java-rest-binding library, i am doing batch insertions by 500 cypher queries, Currently I have at most 200k nodes and 1.4m relationships stored in my graph.

With my current data i am already experiencing request timeouts during insertion, I was wondering if there are any configurations that could make the inserting of batch requests faster.

Or maybe some improvements to the query I am currently using Also here is a sample query being used,

MERGE (firstNode {id:'ABC'})
ON CREATE SET firstNode.type="RINGCODE", firstNode.created = 100, firstNode:rbt
ON MATCH SET firstNode.type="RINGCODE", firstNode:rbt 
MERGE (secondNode{id:'RBT-TC664'}) 
WITH firstNode, secondNode OPTIONAL MATCH firstNode - [existing:`sku`] - () 
DELETE existing 
CREATE UNIQUE p = (firstNode)-[r:`sku`]-(secondNode) RETURN p;
1
neo4jcypher

1 Answers

0
votes
  1. Use labels
  2. create an index or unique constraint for the label + property (id)
  3. represent types with labels instead

Otherwise Neo4j has to scan all nodes to find out if the node you want to merge is already in the database.

If you don't need the uniqueness, you can also use create which doesn't check but just creates and doesn't slow down.

What does rbt stand for?

create constraint on (n:Rbt) assert n.id is unique;

MERGE (firstNode:Rbt:RingCode {id:'ABC'})
ON CREATE SET firstNode.created = 100 
MERGE (secondNode:Rbt {id:'RBT-TC664'})
WITH firstNode, secondNode
OPTIONAL MATCH firstNode -[existing:`sku`]- () 
DELETE existing 
MERGE p = (firstNode)-[r:`sku`]-(secondNode) 
RETURN p;