I am getting a following AssertionError when I am trying to import a very large graph 10b+ nodes and relationships. There are potential nodes that have large # of relationships, up to 10m. When I reduce the size of data import (< 10b nodes and up to < 1m relationships on any given node), I am not running into the error. This is a prototype and I am trying to understand whether I am hitting a limit of some sort or if anyone can provide some directions in how to go about debugging this.
I do realize that there are 34b nodes and 34b relationships limit which is not what I am running into.
This is on Windows 64 bit machine.
Thank you.
2015-06-15 15:53:19.501+0000 ERROR [org.neo4j]: Error during import Requested index -2147483648, but length is 2186000000
rangeParams:
[0, 70483559]
[70483559, 70484353]
[140967912, 70483361]
[211451273, 70484569]
[281935842, 70484114]
[352419956, 70481036]
[422900992, 70477816]
[493378808, 70481928]
[563860736, 70479956]
[634340692, 70477857]
[704818549, 70484071]
[775302620, 70480142]
[845782762, 70487626]
[916270388, 70483085]
[986753473, 70487203]
[1057240676, 70486084]
[1127726760, 70486862]
[1198213622, 70484457]
[1268698079, 70484821]
[1339182900, 70484610]
[1409667510, 70487647]
[1480155157, 70486347]
[1550641504, 70488383]
[1621129887, 70485274]
[1691615161, 70486931]
[1762102092, 70485245]
[1832587337, 70486573]
[1903073910, 70482433]
[1973556343, 70485091]
[2044041434, 70486198]
[2114527632, 70626133]
bucketRange:
661273
674489
687702
700955
714149
724039
730444
736836
743243
749615
756022
762420
768829
775192
781600
789228
800643
812047
823462
834870
846279
1058186
1077351
1096513
1114991
1125762
1136537
1147309
1158082
1168852
8388608
bucketIndex:
70483559
70484353
70483361
70484569
70484114
70481036
70477816
70481928
70479956
70477857
70484071
70480142
70487626
70483085
70487203
70486084
70486862
70484457
70484821
70484610
70487647
70486347
70488383
70485274
70486931
70485245
70486573
70482433
70485091
70486198
32956017
java.lang.RuntimeException: Requested index -2147483648, but length is 2186000000
rangeParams:
[0, 70483559]
[70483559, 70484353]
[140967912, 70483361]
[211451273, 70484569]
[281935842, 70484114]
[352419956, 70481036]
[422900992, 70477816]
[493378808, 70481928]
[563860736, 70479956]
[634340692, 70477857]
[704818549, 70484071]
[775302620, 70480142]
[845782762, 70487626]
[916270388, 70483085]
[986753473, 70487203]
[1057240676, 70486084]
[1127726760, 70486862]
[1198213622, 70484457]
[1268698079, 70484821]
[1339182900, 70484610]
[1409667510, 70487647]
[1480155157, 70486347]
[1550641504, 70488383]
[1621129887, 70485274]
[1691615161, 70486931]
[1762102092, 70485245]
[1832587337, 70486573]
[1903073910, 70482433]
[1973556343, 70485091]
[2044041434, 70486198]
[2114527632, 70626133]
bucketRange:
661273
674489
687702
700955
714149
724039
730444
736836
743243
749615
756022
762420
768829
775192
781600
789228
800643
812047
823462
834870
846279
1058186
1077351
1096513
1114991
1125762
1136537
1147309
1158082
1168852
8388608
bucketIndex:
70483559
70484353
70483361
70484569
70484114
70481036
70477816
70481928
70479956
70477857
70484071
70480142
70487626
70483085
70487203
70486084
70486862
70484457
70484821
70484610
70487647
70486347
70488383
70485274
70486931
70485245
70486573
70482433
70485091
70486198
32956017
at org.neo4j.unsafe.impl.batchimport.staging.StageExecution.stillExecuting(StageExecution.java:62)
at org.neo4j.unsafe.impl.batchimport.staging.ExecutionSupervisor.anyStillExecuting(ExecutionSupervisor.java:79)
at org.neo4j.unsafe.impl.batchimport.staging.ExecutionSupervisor.finishAwareSleep(ExecutionSupervisor.java:102)
at org.neo4j.unsafe.impl.batchimport.staging.ExecutionSupervisor.supervise(ExecutionSupervisor.java:64)
at org.neo4j.unsafe.impl.batchimport.staging.ExecutionSupervisors.superviseExecution(ExecutionSupervisors.java:80)
at org.neo4j.unsafe.impl.batchimport.staging.ExecutionSupervisors.superviseDynamicExecution(ExecutionSupervisors.java:59)
at org.neo4j.unsafe.impl.batchimport.ParallelBatchImporter.executeStages(ParallelBatchImporter.java:239)
at org.neo4j.unsafe.impl.batchimport.ParallelBatchImporter.doImport(ParallelBatchImporter.java:160)
at org.neo4j.tooling.ImportTool.main(ImportTool.java:293)
at org.neo4j.tooling.ImportTool.main(ImportTool.java:227)
Caused by: java.lang.AssertionError: Requested index -2147483648, but length is 2186000000
rangeParams:
[0, 70483559]
[70483559, 70484353]
[140967912, 70483361]
[211451273, 70484569]
[281935842, 70484114]
[352419956, 70481036]
[422900992, 70477816]
[493378808, 70481928]
[563860736, 70479956]
[634340692, 70477857]
[704818549, 70484071]
[775302620, 70480142]
[845782762, 70487626]
[916270388, 70483085]
[986753473, 70487203]
[1057240676, 70486084]
[1127726760, 70486862]
[1198213622, 70484457]
[1268698079, 70484821]
[1339182900, 70484610]
[1409667510, 70487647]
[1480155157, 70486347]
[1550641504, 70488383]
[1621129887, 70485274]
[1691615161, 70486931]
[1762102092, 70485245]
[1832587337, 70486573]
[1903073910, 70482433]
[1973556343, 70485091]
[2044041434, 70486198]
[2114527632, 70626133]
bucketRange:
661273
674489
687702
700955
714149
724039
730444
736836
743243
749615
756022
762420
768829
775192
781600
789228
800643
812047
823462
834870
846279
1058186
1077351
1096513
1114991
1125762
1136537
1147309
1158082
1168852
8388608
bucketIndex:
70483559
70484353
70483361
70484569
70484114
70481036
70477816
70481928
70479956
70477857
70484071
70480142
70487626
70483085
70487203
70486084
70486862
70484457
70484821
70484610
70487647
70486347
70488383
70485274
70486931
70485245
70486573
70482433
70485091
70486198
32956017
at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.ParallelSort.sortRadix(ParallelSort.java:180)
at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.ParallelSort.run(ParallelSort.java:69)
at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.EncodingIdMapper.prepare(EncodingIdMapper.java:219)
at org.neo4j.unsafe.impl.batchimport.IdMapperPreparationStep.process(IdMapperPreparationStep.java:50)
at org.neo4j.unsafe.impl.batchimport.staging.LonelyProcessingStep$1.run(LonelyProcessingStep.java:56)
