Load only few samples from large csv file in neo4j

Question

I am new for neo4j db. I've a large csv file which cannot fit in my machine's ram. Before I load all the records in db using USING PERIODIC COMMIT, I want to test my cypher query on the small sample of data. How can I load load just 1000 rows of data and test out my query.

The data has columns in simplified form as [Employee, CompanyName]. I want to create relationship as (:Employee)-[:Employed]->(:Company). The Employee and the CompanyName nodes are already loaded into the database.

Answer 1

You can limit the rows you want to import with:

USING PERIODIC COMMIT
LOAD CSV WITH HEADERS 'file:///yourcsvfile.csv' AS row
WITH row LIMIT 1000
...

and then continue with your usual import Cypher statements. This will read only the first 1000 lines of your file.

Answer 2

Just create a csv file with the first 1000 lines of your file (and then work with that). On Linux/Unix :

head -1000 yourinputfile.csv > output1000.csv

On Windows (powershell) :

Get-Content "yourinputfile.csv" | select -First 1000 | Out-File "output1000.csv"

Hope this helps.

Regards, Tom

Answer 3

If the CSV dataset records are less than 10 million then use the LOAD CSV Method else use the bulk upload method.

One of optimise code to load and read the CSV is :

:auto USING PERIODIC COMMIT 1000
LOAD CSV WITH HEADERS FROM 'file:///file_name.csv' AS row
WITH row LIMIT 10000

This query first loads 1000 records in memory, execute it and then load a set of another 1000 records till the 100,00 records.

Note : Make sure that you place file_name.csv in the import folder.