Cannot start a neo4j cluster with 3 instances on different machines

Question

I am trying to start a neo4j cluster with 3 instances on 3 diffenrent machines, but the 3rd instance never get started. And it works well in single instance mode.

version: neo4j-enterprise-2.2.0-M01

environment:JDK7,linux-redhat-2.6.18,64bit

The configuration is as below:

instance 1:

neo4j-server.properties:

org.neo4j.server.webserver.address=<ip:1>
org.neo4j.server.database.mode=HA
...

neo4j.properties:

online_backup_server==<ip:1>:6362
ha.initial_hosts=<ip:1>:5001,<ip:2>:5001,<ip:3>:5001
ha.cluster_server=<ip:1>:5001
ha.server=<ip:1>:6001
...

instance 2:

neo4j-server.properties:

org.neo4j.server.webserver.address=<ip:2>
org.neo4j.server.database.mode=HA
...

neo4j.properties:

online_backup_server==<ip:2>:6362
ha.initial_hosts=<ip:1>:5001,<ip:2>:5001,<ip:3>:5001
ha.cluster_server=<ip:2>:5001
ha.server=<ip:2>:6001
...

instance 3:

neo4j-server.properties:

org.neo4j.server.webserver.address=<ip:3>
org.neo4j.server.database.mode=HA
...

neo4j.properties:

online_backup_server==<ip:3>:6362
ha.initial_hosts=<ip:1>:5001,<ip:2>:5001,<ip:3>:5001
ha.cluster_server=<ip:3>:5001
ha.server=<ip:3>:6001
...

And when I started these 3 instance one by one, I got log as below:

--------------terminal------------

WARNING: Max 8196 open files allowed, minimum of 40 000 recommended. See the Neo4j manual.
Using additional JVM arguments:  -server -XX:+DisableExplicitGC -            Dorg.neo4j.server.properties=conf/neo4j-server.properties -  Djava.util.logging.config.file=conf/logging.properties -  Dlog4j.configuration=file:conf/log4j.properties -XX:+UseConcMarkSweepGC -  XX:+CMSClassUnloadingEnabled -XX:MaxDirectMemorySize=2G -XX:-OmitStackTraceInFastThrow
Starting Neo4j Server...WARNING: not changing user
 HA instance started in process [16030]. Will be operational once connected to peers.     See /sbclocal/apps/f35/neo4j/neo4j-enterprise-2.2.0-M01/data/log/console.log for current status.

----------------console.log---------------------

2014-12-12 05:53:17.994+0000 INFO  [Cluster] Write transactions to database disabled
2014-12-12 05:53:19.399+0000 INFO  [Cluster] Attempting to join cluster of [<ip:1>:5001, <ip:2>:5001, <ip:3>:5001]

----------------neo4j.0.0.log--------------------

Dec 11, 2014 7:21:48 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.18.1 02/19/2014 03:28 AM'

For the 3rd instance, the console log just stucks at "Write transactions to database disabled", and then after 1 or 2 minutes, the instance just become down, without errors or exceptions.

And I cannot access the server by ip:7474 UI.

Could anyone give me some infomation about this please? Did I do something wrong? Any answer is appriciated

Can the machine reach / see each other? Are the ports open between them? No firewall rules? — Michael Hunger
@MichaelHunger Thanks for your help, the problem solved. Exception found in data/graph.db/message.log, you can see my answer for the details. — zxz

zxz zxz · Accepted Answer · 2014-12-15T01:17:12

Problem solved. I found a log file in data/graph.db/message.log, and there is Port already in use exception

I really didnt know there is a message.log , I didnt find it in the neo4j documentation, I have to complain about the documentation, it is so confusing and not organized.....

Cannot start a neo4j cluster with 3 instances on different machines

1 Answers