Performance issues building graph with py2neo transactions

Question

I'm trying to build a full binary tree of variable depth using py2neo. I've been trying to use py2neo transactions to send create statements to the server, but the runtime awful.

Building a tree with depth 8 (255 nodes) takes about 16.7 seconds - the vast majority of that time is spent while the transaction is committing (if I Transaction.process() before committing then the processing takes the bulk of the runtime). What could be the issue? The cypher statements are each just a single Match and a node + relationship Create.

Here's the function which builds the tree

def buildBinaryTree(self):
    depth = 1
    tx = self.graph.begin()
    g = self.graph
    leaves = []
    leaves.append("Root")
    tx.run('CREATE(n {ruleName: "Root"})')
    timeSum = 0
    while depth < self.scale:
        newLeaves = []
        for leaf in leaves:
            leftName = leaf + 'L'
            rightName = leaf + 'R'
            newLeaves.append(leftName)
            newLeaves.append(rightName)
            start = timer()

            statement = ('MATCH(n {ruleName: "' + leaf + '"}) '
                        'WITH n CREATE (n)-[:`LINKS TO`]'
                        '->({ruleName: "' + leftName + '"})')
            tx.run(statement)
            statement = ('MATCH(n {ruleName: "' + leaf + '"}) '
                        'WITH n CREATE (n)-[:`LINKS TO`]'
                        '->(m {ruleName: "' + rightName + '"})')
            tx.run(statement)
            end = timer()
            timeSum += (end - start)
        leaves = newLeaves
        depth += 1
        print("Depth = " + str(depth))

    print(timeSum)
    start = timer()
    print("Processing...")
    tx.process()
    print (timer() - start)
    print("Committing...")
    tx.commit()
    print("Committed")
    print (timer() - start)

And the output with scale = 8

building tree...
Depth = 2
Depth = 3
Depth = 4
Depth = 5
Depth = 6
Depth = 7
Depth = 8
0.009257960999775605
Processing...
16.753949095999815
Committing...
Committed
17.28687257200022

Please include the code you are using so others can help figure out the problem. — manassehkatz-Moving 2 Codidact

cybersam cybersam · Accepted Answer · 2017-08-04T01:29:06

Try making these changes to ensure that you are truly running in a single transaction and not unnecessarily delaying committing your updates:

Change tx.run to tx.append. That will enqueue each Cypher statement instead of immediately executing it.
Delete tx.process(), since you have no subsequent Cypher statements to enqueue.

Ideally, you should also be passing parameters instead of modifying your Cypher statements every time through the loop. However, that would not be responsible for the slowness you are experiencing.

Performance issues building graph with py2neo transactions

1 Answers