1
votes

Please, I need your kind help.

I am struggling since last week trying to configure RabbitMQ cluster on windows servers. I reinstalled RabbitMQ 3.7.8 and ErlangOTP 21-1 for more than 15 times trying to figure out the issue and fix it, but unfortunately, all of my attempts failed.

My environment: Three virtual machines with OS: windows server 2012

Environment Variable setting:

•   ERLANG_HOME = C:\Program Files\erl10.1
•   path: C:\Program Files\Handle
•   %RABBITMQ_BASE% = c:\RabbitMQ
•   %RABBITMQ_CONFIG_FILE% = %RABBITMQ_BASE%
•   RABBITMQ_NODENAME = masternode
•   Server names: RabbitMQ01 “masternode” , RabbitMQ02, RabbitMQ03
•   Adding IP & hots name “masternode” in the host file for name resolution (C:\Windows\System32\drivers\etc\hosts)
•   Setting RabbitMQ configuration file “advanced.config” in c:\users\USERNAME\AppData\Roaming\RabbitMQ” to enable users to login to RabbitMQ management remotely by adding the following setting
[
{rabbit,
  [
   {tcp_listeners, [5672]},
   {loopback_users, []},

   {num_tcp_acceptors, 100}
  ]
  }
].

Installing steps: (for each server)

1. After setting the environment variables, I restarted the machine. Then
2. Install ErlangOTP 21-1
3. Install RabbitMQ 3.7.8 server
4. Enable plugins - rabbitmq-plugins.bat enable rabbitmq_management 

After completing the installation of the above servers, I start configuring the cluster

  1. First, I verified that masternode is reachable from RabbitMQ02, 02 by pinging Rabbit@masternode and it’s successful.
  2. Stopped RabbitMQ server and App by using the following command

    rabbitmq-service stop rabbitmqctl stop_app

    1. Copy .erlang.cookie from masternode “path=c:\windows\system32\config\ systemprofile” to RabbitMQ02 “path= c:\windows\system32\config\ systemprofile” and “path=c:\users\USERNAME”.
  3. Start RabbitMQ service & App by executing the below commands

rabbitmqctl start_app rabbitmq-service start

  1. Stop App again by executing this command “rabbitmqctl stop_app”
  2. I start joining RabbitMQ02 to masternode rabbitmqctl join_cluster rabbit@masternode (small case)

Once I reach this point, I’m getting the following error on RabbitMQ02.

C:\Program Files\RabbitMQ Server\rabbitmq_server-3.7.8\sbin>rabbitmqctl join_cluster rabbit@masternode Error: unable to perform an operation on node 'rabbit@rhrabbitmq02'. Please see diagnostics information and suggestions below.

Most common reasons for this are:

  • Target node is unreachable (e.g. due to hostname resolution, TCP connection o r firewall issues)
  • CLI tool fails to authenticate with the server (e.g. due to CLI tool's Erlang cookie not matching that of the server)
  • Target node is not running

In addition to the diagnostics info below:

  • See the CLI, clustering and networking guides on http://rabbitmq.com/document ation.html to learn more
  • Consult server logs on node rabbit@rhrabbitmq02

DIAGNOSTICS

attempted to contact: [rabbit@rhrabbitmq02]

rabbit@rhrabbitmq02: * connected to epmd (port 4369) on rhrabbitmq02 * epmd reports node 'rabbit' uses port 25672 for inter-node and CLI tool traff ic * TCP connection succeeded but Erlang distribution failed

  • Authentication failed (rejected by the remote node), please check the Erlang cookie

Current node details: * node name: rabbitmqcli41@rhrabbitmq02 * effective user's home directory: C:\Users\admin * Erlang cookie hash: pndBksvQokXyiSqBdnne/g==

masternode Status:

C:\Program Files\RabbitMQ Server\rabbitmq_server-3.7.8\sbin>rabbitmqctl status
Status of node masternode@rhrabbitmq01 ...
[{pid,4008},
 {running_applications,
     [{rabbitmq_management,"RabbitMQ Management Console","3.7.8"},
      {rabbitmq_management_agent,"RabbitMQ Management Agent","3.7.8"},
      {rabbitmq_web_dispatch,"RabbitMQ Web Dispatcher","3.7.8"},
      {rabbit,"RabbitMQ","3.7.8"},
      {amqp_client,"RabbitMQ AMQP Client","3.7.8"},
      {rabbit_common,
          "Modules shared by rabbitmq-server and rabbitmq-erlang-client",
          "3.7.8"},
      {os_mon,"CPO  CXC 138 46","2.4.6"},
      {cowboy,"Small, fast, modern HTTP server.","2.2.2"},
      {mnesia,"MNESIA  CXC 138 12","4.15.5"},
      {cowlib,"Support library for manipulating Web protocols.","2.1.0"},
      {lager,"Erlang logging framework","3.6.3"},
      {syslog,"An RFC 3164 and RFC 5424 compliant logging framework.","3.4.3"},
      {ranch_proxy_protocol,"Ranch Proxy Protocol Transport","1.5.0"},
      {jsx,"a streaming, evented json parsing toolkit","2.8.2"},
      {ranch,"Socket acceptor pool for TCP protocols.","1.5.0"},
      {ssl,"Erlang/OTP SSL application","9.0.2"},
      {public_key,"Public key infrastructure","1.6.2"},
      {asn1,"The Erlang ASN1 compiler version 5.0.7","5.0.7"},
      {inets,"INETS  CXC 138 49","7.0.2"},
      {xmerl,"XML parser","1.3.18"},
      {recon,"Diagnostic tools for production use","2.3.2"},
      {crypto,"CRYPTO","4.3.3"},
      {goldrush,"Erlang event stream processor","0.1.9"},
      {compiler,"ERTS  CXC 138 10","7.2.5"},
      {syntax_tools,"Syntax tools","2.1.6"},
      {sasl,"SASL  CXC 138 11","3.2.1"},
      {stdlib,"ERTS  CXC 138 10","3.6"},
      {kernel,"ERTS  CXC 138 10","6.1"}]},
 {os,{win32,nt}},
 {erlang_version,
     "Erlang/OTP 21 [erts-10.1] [64-bit] [smp:2:2] [ds:2:2:10] [async-threads:64
]\n"},
 {memory,
     [{connection_readers,0},
      {connection_writers,0},
      {connection_channels,0},
      {connection_other,29212},
      {queue_procs,0},
      {queue_slave_procs,0},
      {plugins,1192744},
      {other_proc,26188552},
      {metrics,195524},
      {mgmt_db,337648},
      {mnesia,75088},
      {other_ets,2748464},
      {binary,181696},
      {msg_index,27840},
      {code,27573597},
      {atom,1131721},
      {other_system,9846202},
      {allocated_unused,9377056},
      {reserved_unallocated,0},
      {strategy,rss},
      {total,[{erlang,69528288},{rss,78905344},{allocated,78905344}]}]},
 {alarms,[]},
 {listeners,
     [{clustering,25672,"::"},
      {amqp,5672,"::"},
      {amqp,5672,"0.0.0.0"},
      {http,15672,"::"},
      {http,15672,"0.0.0.0"}]},
 {vm_memory_calculation_strategy,rss},
 {vm_memory_high_watermark,0.4},
 {vm_memory_limit,3435785420},
 {disk_free_limit,50000000},
 {disk_free,86038073344},
 {file_descriptors,
     [{total_limit,8092},
      {total_used,2},
      {sockets_limit,7280},
      {sockets_used,0}]},
 {processes,[{limit,1048576},{used,399}]},
 {run_queue,1},
 {uptime,6087},
 {kernel,{net_ticktime,60}}]
1
RabbitMQ is not running on masternode - it's right there in the output. On masternode, what is the output of rabbitmqctl status?Luke Bakken
After installing & uninstalling RabbitMQ many times I noticed that the cookie file name doesn’t look normal, the file name look like this “.erlang” instead of “.erlang.cookie” I don’t know WHY?user3885917
I'm sorry, I copied the meassge where I uninstall RabbitMQ from masternode. Now I reinstall RabbitMQ and copy the cookie to RabbitMQ02, then I tried start App to see if it's going work, but it did not. This is the error message.user3885917
Message updated to reflect the actual error message.user3885917
After the join command failuer, I couldn't start RabbitMQ app in RabbitMQ02 or perform the join with the masternode.user3885917

1 Answers

1
votes

Problem solution: setting RABBITMQ_NODENAME incorrectly “masternode” instead of “rabbit@masternode”, Error in command execution sequence which crucial thing to perform (when joining the cluster - stop_app first then service stop), and restarting the service after changing the configuration instead of reinstalling the rabbitmq service.

I will be happy to share all of the steps details if needed for installing, configuring rabbitmq, and configuring the cluster on windows based on my setup scenario. Best regards.