Possible indexing bug in Neo4j 1.8.3?

0
votes

Environment: Neo4j Community 1.8,2, Node.js 0.10.22, Debian squeeze, JDK 1.6.x

The problem I'm about to describe is very spurious and we are at a loss to figure out what in our code could be causing it. So this is a shot in the dark...

All of our Nodes are assigned a GUID property on creation via TransactionEventHandler plugin unless they have an existing GUID property. We have auto-indexing enabled for this GUID property. This seems to work fine. The majority of our queries are GUID-based. That is, we often find Nodes by GUID as all or part of the query. We've noticed that rarely an existing Node with guidA is overwritten with the properties of a just-created Node with guidB. Note that in this case, the GUIDs were actually generated by a foreign system (we're importing users from one system into another). We can see this happening because we keep a version history for each GUID. And we can see at the time that this problem occurs both guidA and guidB share the same Neo4j node id. It also might be the case that a Node with guidB had been created and then deleted some time in the past. We have to do more experimentation to confirm this.

One hypotheses is that:

  1. the node with guidB was created in the past and had Neo4j id = 1234.
  2. It was then deleted which allowed id 1234 to be reused at some time in the future. However, the guidB --> 1234 record still existed in the index.
  3. The node with guidA was then created and was given Neo4j id 1234.
  4. The user with guidB was then re-imported into the system, looked up by GUID, and because the original record in the index still remained, the node with id 1234 was found.
  5. The properties of the node with id n were then overwritten with guidB's user properties.

The only reason I came up with this is because I know that the Lucene records are not immediately deleted when the associated node is deleted. Again, this happens infrequently and the key may be the deletion of the node.

Any possibility that this is an indexing bug?

1
neo4j
When you delete a node,do you also delete it manually from the index? - Luanne
No. I rely on auto-indexing to take care of that. - bjlevine

1 Answers

2
votes

This issue with auto-indexing was fixed at some point.

It only happens across server restarts after the deletion and before the new node is created, that's why it is so rarely.

What you can do is to query the index for the newly deleted GUID then it will be removed. For safekeeping you can also add a check that compares the GUID of the node returned from the index with the GUID searched for.

Probably a good idea to have a job go over your data and check the index / re-index the data by re-setting the guid property.

And as it is a GUID probably use the unique node creation features with the GUID to create the nodes in the first place?