How to format output (Remove session data) of custom cypher query with Neo4j::Session.query

Question

I'm using Neo4jrb/Neo4j-core for custom cypher querying in my Rails application. I use this output as json. However, everytime I run a query

Eg:

output_hash = Neo4j::Session.query("Match (n) return n limit 1;")

Along with the results I also get a lot of unwanted information about the session itself.

The following is included for EACH node.

      "session": {
    "connection": {
      "parallel_manager": null,
      "headers": {
        "Content-Type": "application/json",
        "User-Agent": "neo4j-gem/7.0.6 (https://github.com/neo4jrb/neo4j)"
      },
      "params": {},
      "options": {
        "params_encoder": null,
        "proxy": null,
        "bind": null,
        "timeout": null,
        "open_timeout": null,
        "boundary": null,
        "oauth": null
      },
      "ssl": {
        "verify": null,
        "ca_file": null,
        "ca_path": null,
        "verify_mode": null,
        "cert_store": null,
        "client_cert": null,
        "client_key": null,
        "certificate": null,
        "private_key": null,
        "verify_depth": null,
        "version": null
      },
      "default_parallel_manager": null,
      "builder": {
        "handlers": [
          {
            "name": "Faraday::Request::BasicAuthentication",
            "args": [
              "neo4j",
              "root"
            ],
            "block": null
          },
          {
            "name": "FaradayMiddleware::MultiJson::EncodeJson",
            "args": [],
            "block": null
          },
          {
            "name": "FaradayMiddleware::MultiJson::ParseJson",
            "args": [
              {
                "symbolize_keys": true,
                "content_type": "application/json"
              }
            ],
            "block": null
          },
          {
            "name": "Faraday::Adapter::NetHttpPersistent",
            "args": [],
            "block": null
          }
        ],
        "app": {
          "header_value": "Basic bmVvNGo6cm9vdA==",
          "app": {
            "app": {
              "app": {
                "app": {}
              },
              "options": {
                "symbolize_keys": true,
                "content_type": "application/json"
              },
              "content_types": [
                "application/json"
              ]
            }
          }
        }
      },
      "url_prefix": {
        "scheme": "http",
        "user": null,
        "password": null,
        "host": "localhost",
        "port": 7474,
        "path": "/",
        "query": null,
        "opaque": null,
        "fragment": null,
        "parser": {
          "regexp": {
            "SCHEME": "(?-mix:\\A[A-Za-z][A-Za-z0-9+\\-.]*\\z)",
            "USERINFO": "(?-mix:\\A(?:%\\h\\h|[!$&-.0-;=A-Z_a-z~])*\\z)",
            "HOST": "(?-mix:\\A(?:(?<IP-literal>\\[(?:(?<IPv6address>(?:\\h{1,4}:){6}(?<ls32>\\h{1,4}:\\h{1,4}|(?<IPv4address>(?<dec-octet>[1-9]\\d|1\\d{2}|2[0-4]\\d|25[0-5]|\\d)\\.\\g<dec-octet>\\.\\g<dec-octet>\\.\\g<dec-octet>))|::(?:\\h{1,4}:){5}\\g<ls32>|\\h{,4}::(?:\\h{1,4}:){4}\\g<ls32>|(?:(?:\\h{1,4}:)?\\h{1,4})?::(?:\\h{1,4}:){3}\\g<ls32>|(?:(?:\\h{1,4}:){,2}\\h{1,4})?::(?:\\h{1,4}:){2}\\g<ls32>|(?:(?:\\h{1,4}:){,3}\\h{1,4})?::\\h{1,4}:\\g<ls32>|(?:(?:\\h{1,4}:){,4}\\h{1,4})?::\\g<ls32>|(?:(?:\\h{1,4}:){,5}\\h{1,4})?::\\h{1,4}|(?:(?:\\h{1,4}:){,6}\\h{1,4})?::)|(?<IPvFuture>v\\h+\\.[!$&-.0-;=A-Z_a-z~]+))\\])|\\g<IPv4address>|(?<reg-name>(?:%\\h\\h|[!$&-.0-9;=A-Z_a-z~])*))\\z)",
            "ABS_PATH": "(?-mix:\\A\\/(?:%\\h\\h|[!$&-.0-;=@-Z_a-z~])*(?:\\/(?:%\\h\\h|[!$&-.0-;=@-Z_a-z~])*)*\\z)",
            "REL_PATH": "(?-mix:\\A(?:%\\h\\h|[!$&-.0-;=@-Z_a-z~])+(?:\\/(?:%\\h\\h|[!$&-.0-;=@-Z_a-z~])*)*\\z)",
            "QUERY": "(?-mix:\\A(?:%\\h\\h|[!$&-.0-;=@-Z_a-z~\\/?])*\\z)",
            "FRAGMENT": "(?-mix:\\A(?:%\\h\\h|[!$&-.0-;=@-Z_a-z~\\/?])*\\z)",
            "OPAQUE": "(?-mix:\\A(?:[^\\/].*)?\\z)",
            "PORT": "(?-mix:\\A[\\x09\\x0a\\x0c\\x0d ]*\\d*[\\x09\\x0a\\x0c\\x0d ]*\\z)"
          }
        }
      },
      "proxy": null
    },
    "resource_data": {
      "extensions": {},
      "node": "http://localhost:7474/db/data/node",
      "relationship": "http://localhost:7474/db/data/relationship",
      "node_index": "http://localhost:7474/db/data/index/node",
      "relationship_index": "http://localhost:7474/db/data/index/relationship",
      "extensions_info": "http://localhost:7474/db/data/ext",
      "relationship_types": "http://localhost:7474/db/data/relationship/types",
      "batch": "http://localhost:7474/db/data/batch",
      "cypher": "http://localhost:7474/db/data/cypher",
      "indexes": "http://localhost:7474/db/data/schema/index",
      "constraints": "http://localhost:7474/db/data/schema/constraint",
      "transaction": "http://localhost:7474/db/data/transaction",
      "node_labels": "http://localhost:7474/db/data/labels",
      "neo4j_version": "3.0.1"
    },
    "resource_url": "http://localhost:7474/db/data/"
  }

The session data seems to contain a lot of sensitive information and so I can't directly use the output.

I'd like to avoid this and just get simple, essential, output to my cypher queries and no extra information. Is there a config setting that I can use to accomplish this?

Thank you.

Update: I'm actually using Neo4jrb/Neo4j (which includes Neo4j-core) as a ActiveModel replacement for few basic tasks in this project. And custom queries for the main tasks.

Further inspection:

As long as an ActiveNode model is defined, I get a clear response without any extra(session) data. But only the properties defined in the model file are returned.

class User
    include Neo4j::ActiveNode

    property :name, type: String
    property :email, type: String
end

This is working fine for the User model, any properties not declared in the model file(but present in the database) are ignored by default according to the documentation.

My problem: My graph has certain nodes that will have many different property keys that cannot be hardcoded beforehand into the model file (Reason for using column-less DB). I cannot declare all the properties of these nodes in the model.

So I'm trying to do a custom cypher query as Neo4j::Session.query(q) But the JSON response is too cluttered with session data haphazardly. Processing this response with code is resulting in a convoluted code because there doesn't seem to be a consistent way the session data is included between results(nodes, edges, aggregators.)

I'm looking for a solution or a way out.

I'm not exactly sure why you're getting all of that data, but there is a setting in the upcoming 8.0 which will allow you to change the way things are wrapped, including returning hash representations of the results — Brian Underwood
Sorry, I needed to go offline quickly last night. You say that all of that data is included for each node. How are you getting the data? It looks like the JSON response, but then the neo4j / neo4j-core gems wrap nodes / relationships in objects. — Brian Underwood
Thanks a lot for getting back Brian. I've updated the post with details. Yes it is a JSON response. When a model is declared it gives me declared properties and no clutter. When there is no model, I get all properties but also the session data clutter. — Nein
I'm sorry, a correction. I get an Enumerator. Which is converted into an Array of Structs by the to_a method. [#<struct n=CypherNode xx (70157506237380)>,...] is the array. In my application I directly do an render json: Enumerator, which converts the data to json. Furthermore, the json includes data from Faraday too. Maybe if some variables are getting passed along to Neo4j-core, it is including it with each row data? — Nein
I could be out of my depth here. But going through the source code, I find this line peculiar and a potential cause of my problem. Neo4j-core CypherResponse Class — Nein

Nein Nein · Accepted Answer · 2016-06-05T16:10:08

Thanks Brian for your help. To summarise:

The session key in the output is because Neo4j-core returns the respective object(node/edge) whenever possible, refer this line here in the source code. Both the CypherNode and CypherRelationship classes have the @session variable and when their objects are converted to JSON/Hash/YAML, the session data is being included auto.

One option is to write your own custom methods and extend the class to give cleaner JSON output. Brian mentions that upcoming 8.0 will give syntax options in the response. Or you can write your custom code in your app to process the response from Neo4j-core.

I finally ended up doing the latter as below:

class TestController < ApplicationController

    def test
        q = "Match (n) return n;"
        data = Neo4j::Session.query(q)
        output = clean_response(data)

        render json: output
    end

    private

    def clean_response(data)
        # data = Enumerator
        data = data.map(&:to_h)
        # data = [ { x: => CypherXxxx <Object>},...   ]
        data.map! do |entry|
            hash = JSON.parse(entry.to_json)
            # Clean json of the entry
            entry = delete_keys(hash)
        end
    end

    def delete_keys(hash)
        # Recursive func to delete session/response_hash from a hash of any depth 
        if(hash.class == Hash)
            hash.delete("session") if hash.keys.include?("session")
            hash.delete("response_hash") if hash.keys.include?("response_hash")
            hash.each_value{|value|
              value = delete_keys(value)
            }
        end
    end
end

How to format output (Remove session data) of custom cypher query with Neo4j::Session.query

2 Answers