How to export csv file of large resultset using cypher in Neo4j in the browser?

Question

I am using Neo4j on my browser on Ubuntu. I got over 1 million nodes and I want to export them as csv file.

When return data size is small like "match n return n limit 3" there is a big fat "download csv" button I could use. But when it comes to big result set like over 1000 the shell just says "Resultset too large(over 1000 rows)" and the button doesnt show up.

How can I export csv files for large resultset?

Answer 1

You can also use my shell extensions to export cypher results to CSV.

See here: https://github.com/jexp/neo4j-shell-tools#cypher-import

Just provide an -o output.csv file to the import-cypher command.

Answer 2

Well, I just used linux shell to do all the job.

neo4j-shell -file query.cql | sed 's/|/;/g' > myfile.csv

In my case, I had also to convert from UTF-8 to ISO-8859-1 so I typed:

neo4j-shell -file query.cql | sed 's/|/;/g' | iconv -f UTF-8 -t ISO-8859-1 -o myfile.csv

PS: sed performs the replace: 's/|/;/g' means, substitute (s) all "|" to ";" even though there is more than one per line (g)

Hope this can help.
Regards

Answer 3

We followed the approach below using mentioned. It works very well for us. data is formatted properly in csv format.

https://github.com/jexp/neo4j-shell-tools#cypher-import

import-cypher command from neo4J shell.

neo4j-sh (?)$ import-cypher -o test.csv MATCH (m:TDSP) return m.name

Answer 4

I know this is an old post, but maybe this will help someone else. For anyone using Symfony Framework, you can make a pretty simple controller to export Neo4j Cypher Queries as CSV. This uses the Graphaware NEO4J PHP OGM ( https://github.com/graphaware/neo4j-php-ogm ) to run raw cypher queries. I guess this can also easily be implemented without Symfony only using plain PHP.

Just make a form (with twig if you want):

  <form action="{{ path('admin_exportquery') }}" method="get">
    Cypher:<br>
    <textarea name="query"></textarea><br>
    <input type="submit" value="Submit">
  </form>

Then set up the route of "admin_exportquery" to point to the controller. And add a controller to handle the export:

public function exportQueryAction(Request $request)
    {
        $query = $request->query->get('query');
        $em = $this->get('neo4j.graph_manager')->getClient();

        $response = new StreamedResponse(function() use($em,$query) {
            $result = $em->getDatabaseDriver()->run($query);
            $handle = fopen('php://output', 'w');
            fputs( $handle, "\xEF\xBB\xBF" );
            $header = $result->getRecords()[0]->keys();

            fputcsv($handle, $header);
            foreach($result->getRecords() as $r){
                fputcsv($handle, $r->values());
            }

            fclose($handle);
        });

        $response->headers->set('Content-Type', 'application/force-download');
        $response->headers->set('Cache-Control', 'no-store, no-cache');
        $response->headers->set('Content-Disposition','attachment; filename="export.csv"');

        return $response;

    }

This lets you download a CSV with utf-8 characters directly from your browser and gives you all the freedom of Cypher.

IMPORTANT: This has no query check what so ever and it is a very good idea to set up appropriate security or query check before using :)

Answer 5

The cypher-shell:

https://neo4j.com/docs/operations-manual/current/tools/cypher-shell/

that is included in the latest versions of neo4j does this easily:

cat query.cql | cypher-shell > answer.csv

Answer 6

The limit 1000 is due to the browser MaxRows setting. You can change it to e.g. 10000 and thus be able to download those 10000 in one go via the download/export csv button described in the original post. On my laptop, the limit for the download button is somewhere between 10000 and 20000. By setting the MaxRows limit to 50000 or 300000, I have been able to get the data on screen after waiting a while. Manual select, copy, and paste works. However, doing anything else than closing the browser after each query has not been possible as the browser becomes very slow.

Answer 7

There is another option to export the data as CSV using the cURL command with http/https neo4j transaction/commit endpoint.

here is an example on how to do that.

curl -H accept:application/json -H content-type:application/json \
     -d '{"statements":[{"statement":"MATCH (p1:PROFILES)-[:RELATION]-(p2) RETURN ... LIMIT 4"}]}' \
     http://localhost:7474/db/data/transaction/commit \
     | jq -r '(.results[0]) | .columns,.data[].row | @csv'

and this command uses jq. make sure jq is installed in your machine and this converts the results to the CSV format.

Note: You may need to pass Authorization in the header for authentication.

Just pass -H 'Authorization: Basic XXXXX=' \ to avoid 401

here is the blog with a detailed explanation. https://neo4j.com/blog/export-csv-from-neo4j-curl-cypher-jq/