Neo4j 3.0.0 data migration

3
votes

I've trying to move from Neo4j 2.3.3 to 3.0.0 by using auto-migration function by setting "dbms.allow_format_migration=true" in neo4j.conf (like for all 2.X upgrades).

Unfortunately I obtain some errors without the possibility to access my data :

2016-05-11 16:32:15.684+0000 INFO  Starting...
2016-05-11 16:32:16.576+0000 INFO  Bolt enabled on localhost:7687.
2016-05-11 16:32:16.724+0000 INFO  Starting upgrade of database
2016-05-11 16:32:16.942+0000 INFO  Migrating Indexes (1/3):
2016-05-11 16:32:16.947+0000 INFO    10% completed
2016-05-11 16:32:16.948+0000 INFO    20% completed
2016-05-11 16:32:16.948+0000 INFO    30% completed
2016-05-11 16:32:16.948+0000 INFO    40% completed
2016-05-11 16:32:16.948+0000 INFO    50% completed
2016-05-11 16:32:16.948+0000 INFO    60% completed
2016-05-11 16:32:16.949+0000 INFO    70% completed
2016-05-11 16:32:16.949+0000 INFO    80% completed
2016-05-11 16:32:16.949+0000 INFO    90% completed
2016-05-11 16:32:16.949+0000 INFO    100% completed
2016-05-11 16:32:16.949+0000 INFO  Migrating Legacy indexes (2/3):
2016-05-11 16:32:17.616+0000 ERROR Failed to start Neo4j: Starting Neo4j failed: Component 'org.neo4j.server.database.LifecycleManagingDatabase@768767a1' was successfully initialized, but failed to start. Please see attached cause exception. Starting Neo4j failed: Component 'org.neo4j.server.database.LifecycleManagingDatabase@768767a1' was successfully initialized, but failed to start. Please see attached cause exception.
org.neo4j.server.ServerStartupException: Starting Neo4j failed: Component 'org.neo4j.server.database.LifecycleManagingDatabase@768767a1' was successfully initialized, but failed to start. Please see attached cause exception.
        at org.neo4j.server.exception.ServerStartupErrors.translateToServerStartupError(ServerStartupErrors.java:68)
        at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:217)
        at org.neo4j.server.ServerBootstrapper.start(ServerBootstrapper.java:81)
        at org.neo4j.server.ServerBootstrapper.start(ServerBootstrapper.java:60)
        at org.neo4j.server.CommunityEntryPoint.main(CommunityEntryPoint.java:28)
Caused by: org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.server.database.LifecycleManagingDatabase@768767a1' was successfully initialized, but failed to start. Please see attached cause exception.
        at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:444)
        at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:107)
        at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:189)
        ... 3 more
Caused by: java.lang.RuntimeException: Error starting org.neo4j.kernel.impl.factory.CommunityFacadeFactory, /usr/local/Cellar/neo4j/3.0.0/libexec/data/databases/graph_ito.db
        at org.neo4j.kernel.impl.factory.GraphDatabaseFacadeFactory.newFacade(GraphDatabaseFacadeFactory.java:144)
        at org.neo4j.kernel.impl.factory.CommunityFacadeFactory.newFacade(CommunityFacadeFactory.java:40)
        at org.neo4j.kernel.impl.factory.GraphDatabaseFacadeFactory.newFacade(GraphDatabaseFacadeFactory.java:108)
        at org.neo4j.server.CommunityNeoServer.lambda$static$32(CommunityNeoServer.java:55)
        at org.neo4j.server.database.LifecycleManagingDatabase.start(LifecycleManagingDatabase.java:89)
        at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:434)
        ... 5 more
Caused by: org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.kernel.NeoStoreDataSource@10b461e3' was successfully initialized, but failed to start. Please see attached cause exception.
        at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:444)
        at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:107)
        at org.neo4j.kernel.impl.transaction.state.DataSourceManager.start(DataSourceManager.java:99)
        at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:434)
        at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:107)
        at org.neo4j.kernel.impl.factory.GraphDatabaseFacadeFactory.newFacade(GraphDatabaseFacadeFactory.java:140)
        ... 10 more
Caused by: org.neo4j.kernel.impl.storemigration.StoreUpgrader$UnableToUpgradeException: Failure doing migration
        at org.neo4j.kernel.impl.storemigration.StoreUpgrader.migrateToIsolatedDirectory(StoreUpgrader.java:248)
        at org.neo4j.kernel.impl.storemigration.StoreUpgrader.migrateIfNeeded(StoreUpgrader.java:132)
        at org.neo4j.kernel.impl.storemigration.DatabaseMigrator.migrate(DatabaseMigrator.java:98)
        at org.neo4j.kernel.NeoStoreDataSource.upgradeStore(NeoStoreDataSource.java:554)
        at org.neo4j.kernel.NeoStoreDataSource.start(NeoStoreDataSource.java:433)
        at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:434)
        ... 15 more
Caused by: java.io.IOException: Legacy index migration failed.
        at org.neo4j.kernel.impl.storemigration.participant.LegacyIndexMigrator.migrateLegacyIndexes(LegacyIndexMigrator.java:127)
        at org.neo4j.kernel.impl.storemigration.participant.LegacyIndexMigrator.migrate(LegacyIndexMigrator.java:77)
        at org.neo4j.kernel.impl.storemigration.StoreUpgrader.migrateToIsolatedDirectory(StoreUpgrader.java:240)
        ... 20 more
Caused by: org.neo4j.upgrade.lucene.LegacyIndexMigrationException: Migration of legacy index at path:/usr/local/Cellar/neo4j/3.0.0/libexec/data/databases/graph_ito.db/upgrade/index/lucene/node/Hr failed.
        at org.neo4j.upgrade.lucene.LuceneLegacyIndexUpgrader.upgradeIndexes(LuceneLegacyIndexUpgrader.java:114)
        at org.neo4j.kernel.impl.storemigration.participant.LegacyIndexMigrator.migrateLegacyIndexes(LegacyIndexMigrator.java:121)
        ... 22 more
Caused by: org.apache.lucene.index.IndexFormatTooOldException: Format version is not supported (resource BufferedChecksumIndexInput(MMapIndexInput(path="/usr/local/Cellar/neo4j/3.0.0/libexec/data/databases/graph_ito.db/upgrade/index/lucene/node/Hr/segments_1"))): -11 (needs to be between 1071082519 and 1071082519). This version of Lucene only supports indexes created with release 4.0 and later.
        at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:299)
        at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:493)
        at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:490)
        at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:731)
        at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:683)
        at org.apache.lucene.index.SegmentInfos.readLatestCommit(SegmentInfos.java:490)
        at org.apache.lucene.index.DirectoryReader.listCommits(DirectoryReader.java:257)
        at org.apache.lucene.index.IndexUpgrader.upgrade(IndexUpgrader.java:158)
        at org.apache.lucene.index.IndexUpgrader.main(IndexUpgrader.java:78)
        at org.neo4j.upgrade.lucene.IndexUpgraderWrapper.upgradeIndex(IndexUpgraderWrapper.java:66)
        at org.neo4j.upgrade.lucene.LuceneLegacyIndexUpgrader.upgradeIndexes(LuceneLegacyIndexUpgrader.java:109)
        ... 23 more

Any idea ?

1
neo4jupgrade
There is an existing neo4j issue (github.com/neo4j/neo4j/issues/7042) for what seems to be the same problem, but it was closed by the submitter without an explanation for how it was fixed. You may want to open a new issue (or reopen the old one). – cybersam

1 Answers

1
votes

Referencing from the Neo4j Github Issue , I was able to resolve the issue by removing some files from neo4j index.

I had to remove the ${GRAPH_DATABASE}/index/lucene/node/index/segment* files and then restarting the neo4j server did the trick for me.

You can reference the github issues link for detail. Moreover you can also just delete the files pointed out on the error log as well,

Caused by: java.lang.IllegalArgumentException: This tool was invoked to not delete prior commit points, but the following commits were found: [DirectoryReader.ReaderCommit(segments_3), DirectoryReader.ReaderCommit(segments_5)]

Note : I was migrating from 2.3.0-M03 to 3.0.1.