Neo4j Configuration for 4M Nodes 10M relationship

0
votes

I am new to Neo4j and have made few graph queries with 4M nodes and 10M relationships. Till now i've completely surprised from the performance of my queries. 

SCHEMA
.......

(a:user{data:1})-[:follow]->(:user)-[:next*1..10]-(:activity)

Here user with data:1 is following another 100,000 user. Each of those 100,000 users have 2-8 next nodes(lets say activity of users) attached. Now i want to fetch the activities of users till next level 3 [:next*1..3] . Each activity has property relevance number.

So now i have 100,000 *3 nodes to traverse. 

CYPHER
.......


match (u:user{data:1})-[:follow]-(:user)-[:next*1..3]-(a:activity)
return a order by a.relevance desc limit 50

This query is taking 72000 ms almost every time. Since i am new to Neo4j and i am sure that i haven't done tuning of the OS.

I am using following parameters-

Initial Java Heap Size (in MB)

wrapper.java.initmemory=2000

Maximum Java Heap Size (in MB)

wrapper.java.maxmemory=2456

Default values for the low-level graph engine

neostore.nodestore.db.mapped_memory=25M neostore.relationshipstore.db.mapped_memory=50M neostore.propertystore.db.mapped_memory=90M neostore.propertystore.db.strings.mapped_memory=130M neostore.propertystore.db.arrays.mapped_memory=130M

Please tell me where i am doing wrong. I read all the documentation from neo4j website but the query time didn't improve.

please tell me how can i configure high performing cache? What should i do so that all the graph loads up in memory? When i see my RAM usage , it is always like 1.8 GB out of 4 Gb. I am using enterprise license on windows (Neo4j 2.0). Please help.

1
cypherneo4j

1 Answers

1
votes

You are actually following not 100k * 3 but, 100k * (2-10)^10 meaning 10^15 paths.

More memory in your machine would make a lot of sense, so try to get 8 or more GB.

Then you can increase the heap, e.g. to 6GB:

wrapper.java.initmemory=6000
wrapper.java.maxmemory=6000

neo4j.properties

neostore.nodestore.db.mapped_memory=100M
neostore.relationshipstore.db.mapped_memory=500M
neostore.propertystore.db.mapped_memory=200M     
neostore.propertystore.db.strings.mapped_memory=200M 
neostore.propertystore.db.arrays.mapped_memory=10M

If you want to pull your data through, you would most probably want to inverse your query.

match (a:activity),(u:user{data:1})
with a,u
order by a.relevance 
desc limit 100
match (followed:user)-[:next*1..3]-(a:activity)
where (followed)-[:follow]-(user)
return a 
order by a.relevance 
desc limit 50