Cassandra cqlsh - connection refused

59
votes

I've just started working with Cassandra (datastax), version 2.1.3 and cqlsh version 5.0.1.

Cassandra starts up fine and the cluster is operational instantly.

Cqlsh is not working (on any of the nodes) and emits the following error:

Connection error: ('Unable to connect to any servers', {'127.0.0.1': error(111, "Tried connecting to [('127.0.0.1', 9042)]. Last error: Connection refused")})

I have tried starting cqlsh up with the hosts own ip, other hosts ip, different ports yet the result remains the same - always connection refused.

20
cassandracqlsh
Considering I, and with the help offered here, couldn't solve the issue with 2.1.3 (which admittedly is still in development) I reconfigured the servers with 2.0.13 (stable). Cqlsh still gives a localhost:9160 error (despite the rpc_address being specified as the hosts) but it works by manually specifying the ip when starting it up cqlsh 1.1.2.2 9160 – Chros
Based on the fact that you downgraded to 2.0.13 and it works, I'm guessing that you had some kind of incorrect setting between listen_address, rpc_address and broadcast_rpc_address...the latter setting being new with 2.1.x. I would be interested to know the listen_address and rpc_address settings (not necessarily the values, but how they relate to the 1.1.2.2 IP used with cqlsh). If you try 2.1.x again, set listen_address to your local IP, rpc_address to 0.0.0.0, and broadcast_rpc_address to your external IP. – Aaron
In the case of this setup with 1.1.2.2 being the host IP (host only has an internal IP) the values were: listen_address: 1.1.2.2 rpc_address: 0.0.0.0 broadcast_rpc_address: 1.1.2.2. The broadcast address was required (cassandra logs). Tried most combinations yet none proved fruitful. Ended up having to remove/comment it out for 2.0.13 so it could very well be the broadcast_rpc_address creating the problems. Not sure which other value I could have given it. Wouldn't it have thrown an error anywhere else? – Chros

20 Answers

23
votes

You need to edit cassandra.yaml on the node you are trying to connect to and set the node ip address for rpc_address and listen_address and restart Cassandra.

rpc_address is the address on which Cassandra listens to the client calls.

listen_address is the address on which Cassandra listens to the other Cassandra nodes.

20
votes

Try to change the rpc_address to point to the node's IP instead of 0.0.0.0 and specify the IP while connecting to the cqlsh, as if the IP is 10.0.1.34 and the rpc_port left to the default value 9160 then the following should work: 

cqlsh 10.0.1.34 9160

Or:

cqlsh 10.0.1.34

Also make sure that start_rpc is set to true in /etc/cassandra/cassandra.yaml configuration file.

11
votes

If you check the system.log file for cassandra in /var/log/cassandra, you will see that this problem occurs because the rpc server has not started.

By default, the start_rpc is set to false in the cassandra.yaml file. Set it to start_rpc: true and then try again.

From Cassandra 3.0 onwards, start_rpc is set to true by default. https://docs.datastax.com/en/cassandra/3.0/cassandra/configuration/configCassandra_yaml.html

6
votes

This is because of some pre-configurations in the cassandra. And to fix this, there are following commented lines in the cassandra-env.sh file:

#add this if you’re having trouble connecting:
#JVM_OPTS="$JVM_OPTS -Djava.rmi.server.hostname=<public name>

where you have to start fixing the error.

Open the terminal and start fixing using the following easy steps:

Step-1

use

sudo nano /etc/cassandra/cassandra-env.sh

this command and search for JVM_OPTS="$JVM_OPTS -Djava.rmi.server.hostname= uncomment the line by simply removing # attached in the start of the line.

NOTE: If you have never opened this file to fix this error then you will find JVM_OPTS="$JVM_OPTS -Djava.rmi.server.hostname=<public name>

Step-2 Now replace <public name> with 127.0.0.1

Step-3 Save the file and restart the cassandra using systemctl restart cassandra.service (if you haven't the server is alredy running). OR start the cassandra using systemctl start cassandra.service (if the server is not running).

Step-4 Check the status either by using sudo service cassandra status or systemctl status cassandra.service.

Note: Once check the system monitor whether cassandra is running there or not.

Try cqlsh now. It'll work.

Note: nano is editor you can use other editors you are comfortable with.

4
votes

Look for native_transport_port in /etc/cassandra/cassandra.yaml The default is 9842.

native_transport_port: 9842

For connecting to localhost with cqlsh, this port worked for me.

cqlsh 127.0.0.1 9842

3
votes

It is a good idea to check the cassandra log if even the server is running. I was getting exactly the same message and unable to do anything with that and then I found out that there are errors in the log and the system is actually not working.

Silly, I know, but could happen...

3
votes

Had same problem recently after downgrade from Cassandra 3.0 to Cassandra 2.2 on ArchLinux.

Unlike above solutions my problem wasn't in .cassandra, but version 3.0 left its configuration in /var/lib/cassandra directory.

Following commands solved my problem:

sudo rm -R /var/lib/cassandra
sudo rm -R /var/log/cassandra
sudo rm -R /usr/share/cassandra

Then i installed cassandra and everything worked again :)

3
votes

Try to change the rpc_address to point to the node's IP instead of 0.0.0.0 and specify the IP while connecting to the cqlsh, as if the IP is 10.0.2.64 and the rpc_port left to the default value 9160 then the following should work:

cqlsh 10.0.2.64 9160

OR

cqlsh 10.0.2.64

Also make sure that start_rpc is set to true in /etc/cassandra/cassandra.yaml configuration file.

2
votes

The first and foremost step to diagnose is to check the Cassandra logs.

Find out the process and there will be hints in the command line parameters to tell you where the logs files are.

ps aux | grep cassandra

In my case, it was in default location /etc/cassandra/conf/cassandra.yaml (if you installed the yum package).

Make sure three things are configured and configured the same host/ip.

  1. listen_address - you may leave it blank it will default to hostname/first IP binding with eth0 network card
  2. rpc_address - this is optional, leave blank if you want it to be same as listen_address
  3. seeds - this is a double quoted string of comma-separated list of IP addresses or hostnames. This was default to a hardcoded "127.0.0.1", changing it to the same as list_address and restart of Cassandra service made it work for me.

References:

1
votes

try changing the native_transport_protocol to port 9160 (if it is set to anything other than 9160; it might be pointing to 9042). Check your logs and see on which port cassandra is listening for CQL clients?

1
votes

I was experiencing the same issue with Cassandra 3.11.0, anytime I changed the address of rpc or listen address cqlsh wouldn't work. I had to add the same local ip to seeds

So after much trial and error my working cassandra.yml ended up like this:

class-name: org.apache.cassandra.locator.SimpleSeedProvider
parameters: 
    -seeds: "192.168.0.30"

listen_adress: 192.168.0.30
rpc_address: 192.168.0.30
1
votes

For me it turned out that the service wasn't running at all. Check with 

service cassandra status

If you got the same error as I got, or another type, then messing around with IP addresses won't solve your problem at all.

The error I got:

cassandra dead but pid file exists

Edit: This was the solution for my problem: https://stackoverflow.com/a/46743119/3881406

1
votes

Got into this issue for [cqlsh 5.0.1 | Cassandra 3.11.4 | CQL spec 3.4.4 | Native protocol v4] had to set start_native_transport: true in cassandra.yaml file.

For verification,

  • try opening tailf /var/log/cassandra/system.log file in one-tab
  • update cassandra.yaml
  • restart cassandra sudo service cassandra restart

In logfile is shows.

INFO  [main] 2019-03-15 19:53:06,156 Server.java:156 - Starting listening for CQL clients on /10.139.45.34:9042 (unencrypted)...
0
votes

I was trying to run it on a network drive. So I increased the client_timeout in cqlsh and it worked.

0
votes

Make sure that you also set "seeds" to the address which you provided at "listen_address"

0
votes

Try to telnet to the given address. In my case, there was a firewall in place blocking me.

0
votes

When I installed Cassandra 3.11.1, I came across this problem. I checked the /var/log/cassandra/cassandra.log and found this error Exception encountered during startup....It is a bug and already reported. The original post link https://issues.apache.org/jira/browse/CASSANDRA-14173.

The solution is to downgrade Cassandra to 3.0

  1. download Cassandra rpm

curl -O https://www.apache.org/dist/cassandra/redhat/30x/cassandra-3.0.15-1.noarch.rpm

or

wget https://www.apache.org/dist/cassandra/redhat/30x/cassandra-3.0.15-1.noarch.rpm

  1. rpm -ivh cassandra-3.0.15-1.noarch.rpm
  2. service cassandra start
  3. service cassandra status # check cassandra status

cassandra (pid 2322) is running...

  1. cqlsh # start cassandra
0
votes

cqlsh --cqlversion="3.4.0" Connected to Test Cluster at 127.0.0.1:9042. [cqlsh 5.0.1 | Cassandra 3.0.9 | CQL spec 3.4.0 | Native protocol v4]

try the above command. It works for me.

0
votes

First, you need to check the port 9042 whether this open or not if open then check below things like rpc_address and listen_address. you should set the server own IP address instead 0.0.0.0 or 127.0.0.1.

If all okay then please run below command to connect cqlsh.

cqlsh IP address 9042

If you enable authentication and authorization you have to put user/pass with this command.

0
votes

Check for correct IP address in the cassandra.yaml file. Majority of times the error is due to incorrect IP address of your system also the username and password too.

After doing so initiate cqlsh by the command :-

cqlsh 10.31.79.1 -u cassandra -p cassandra