Deploy cassandra-mesos framework with marathon

Question

I'm trying cassandra-mesos on my private cluster os mesos. I'm using Readme instruction to deploy with marathon but some error occure on wake up task and the framework never was registered:

Stderr output:

I0925 15:55:06.185039  8383 fetcher.cpp:214] Fetching URI 'https://downloads.mesosphere.io/cassandra-mesos/artifacts/0.2.1-SNAPSHOT-589-master-4c6502b0a6/cassandra-mesos-0.2.1-SNAPSHOT-589-master-4c6502b0a6.tar.gz'
I0925 15:55:06.185165  8383 fetcher.cpp:125] Fetching URI 'https://downloads.mesosphere.io/cassandra-mesos/artifacts/0.2.1-SNAPSHOT-589-master-4c6502b0a6/cassandra-mesos-0.2.1-SNAPSHOT-589-master-4c6502b0a6.tar.gz' with os::net
I0925 15:55:06.185180  8383 fetcher.cpp:135] Downloading 'https://downloads.mesosphere.io/cassandra-mesos/artifacts/0.2.1-SNAPSHOT-589-master-4c6502b0a6/cassandra-mesos-0.2.1-SNAPSHOT-589-master-4c6502b0a6.tar.gz' to '/tmp/mesos/slaves/20150925-135709-503717292-5050-2136-S0/frameworks/20150925-144548-503717292-5050-5066-0001/executors/cassandra-dev-test.51620664-63bf-11e5-b350-56847afe9799/runs/e0aaadd9-9c86-47e1-8df8-d2c437f960f4/cassandra-mesos-0.2.0-1.tar.gz'
I0925 15:56:07.968350  8383 fetcher.cpp:78] Extracted resource '/tmp/mesos/slaves/20150925-135709-503717292-5050-2136-S0/frameworks/20150925-144548-503717292-5050-5066-0001/executors/cassandra-dev-test.51620664-63bf-11e5-b350-56847afe9799/runs/e0aaadd9-9c86-47e1-8df8-d2c437f960f4/cassandra-mesos-0.2.0-1.tar.gz' into '/tmp/mesos/slaves/20150925-135709-503717292-5050-2136-S0/frameworks/20150925-144548-503717292-5050-5066-0001/executors/cassandra-dev-test.51620664-63bf-11e5-b350-56847afe9799/runs/e0aaadd9-9c86-47e1-8df8-d2c437f960f4'
I0925 15:56:07.971684  8383 fetcher.cpp:214] Fetching URI 'https://downloads.mesosphere.io/java/jre-7u76-linux-x64.tar.gz'
I0925 15:56:07.971709  8383 fetcher.cpp:125] Fetching URI 'https://downloads.mesosphere.io/java/jre-7u76-linux-x64.tar.gz' with os::net
I0925 15:56:07.971725  8383 fetcher.cpp:135] Downloading 'https://downloads.mesosphere.io/java/jre-7u76-linux-x64.tar.gz' to '/tmp/mesos/slaves/20150925-135709-503717292-5050-2136-S0/frameworks/20150925-144548-503717292-5050-5066-0001/executors/cassandra-dev-test.51620664-63bf-11e5-b350-56847afe9799/runs/e0aaadd9-9c86-47e1-8df8-d2c437f960f4/jre-7u76-linux-x64.tar.gz'
I0925 15:56:51.630692  8383 fetcher.cpp:78] Extracted resource '/tmp/mesos/slaves/20150925-135709-503717292-5050-2136-S0/frameworks/20150925-144548-503717292-5050-5066-0001/executors/cassandra-dev-test.51620664-63bf-11e5-b350-56847afe9799/runs/e0aaadd9-9c86-47e1-8df8-d2c437f960f4/jre-7u76-linux-x64.tar.gz' into '/tmp/mesos/slaves/20150925-135709-503717292-5050-2136-S0/frameworks/20150925-144548-503717292-5050-5066-0001/executors/cassandra-dev-test.51620664-63bf-11e5-b350-56847afe9799/runs/e0aaadd9-9c86-47e1-8df8-d2c437f960f4'
I0925 15:56:51.720883  8381 exec.cpp:132] Version: 0.22.1
I0925 15:56:51.723655  8426 exec.cpp:206] Executor registered on slave 20150925-135709-503717292-5050-2136-S0
2015-09-25 15:56:52,376:8432(0x7f3cb2ffd700):ZOO_INFO@log_env@712: Client environment:zookeeper.version=zookeeper C client 3.4.5
2015-09-25 15:56:52,376:8432(0x7f3cb2ffd700):ZOO_INFO@log_env@716: Client environment:host.name=vcmms.domain.com
2015-09-25 15:56:52,376:8432(0x7f3cb2ffd700):ZOO_INFO@log_env@723: Client environment:os.name=Linux
2015-09-25 15:56:52,376:8432(0x7f3cb2ffd700):ZOO_INFO@log_env@724: Client environment:os.arch=3.19.3-1.el6.x86_64
2015-09-25 15:56:52,376:8432(0x7f3cb2ffd700):ZOO_INFO@log_env@725: Client environment:os.version=#1 SMP Mon Mar 30 13:50:16 EDT 2015
2015-09-25 15:56:52,376:8432(0x7f3cb2ffd700):ZOO_INFO@log_env@733: Client environment:user.name=(null)
2015-09-25 15:56:52,376:8432(0x7f3cb2ffd700):ZOO_INFO@log_env@741: Client environment:user.home=/root
2015-09-25 15:56:52,376:8432(0x7f3cb2ffd700):ZOO_INFO@log_env@753: Client environment:user.dir=/tmp/mesos/slaves/20150925-135709-503717292-5050-2136-S0/frameworks/20150925-144548-503717292-5050-5066-0001/executors/cassandra-dev-test.51620664-63bf-11e5-b350-56847afe9799/runs/e0aaadd9-9c86-47e1-8df8-d2c437f960f4
2015-09-25 15:56:52,376:8432(0x7f3cb2ffd700):ZOO_INFO@zookeeper_init@786: Initiating client connection, host=172.29.6.30:2181 sessionTimeout=10000 watcher=0x7f3cee4ceeb0 sessionId=0 sessionPasswd=<null> context=0x7f3c9c000930 flags=0
2015-09-25 15:56:52,377:8432(0x7f3cb17fa700):ZOO_INFO@check_events@1703: initiated connection to server [172.29.6.30:2181]
2015-09-25 15:56:52,382:8432(0x7f3cb17fa700):ZOO_INFO@check_events@1750: session establishment complete on server [172.29.6.30:2181], sessionId=0x15005d328430014, negotiated timeout=10000

and Stdout output:

Registered executor on vcmms.domain.com
Starting task cassandra-dev-test.51620664-63bf-11e5-b350-56847afe9799
Forked command at 8432
sh -c '$(pwd)/jre*/bin/java $JAVA_OPTS -classpath cassandra-mesos-framework.jar io.mesosphere.mesos.frameworks.cassandra.framework.Main'
Command exited with status 10 (pid: 8432)

Cluster description: mesos masters: 1 mesos slaves: 3 with slave1 4 CPU 6.8 GB RAM 139.9 GB DISK slave2 4 CPU 6.8 GB RAM 139.9 GB DISK slave3 4 CPU 6.8 GB RAM 31.6 GB DISK

If you're thinking about using c* and mesos in production watch this jira issues.apache.org/jira/browse/MESOS-1554 — phact
I don't see any error here, can you share the Mesos master log (at least the relevant bits)? — Michael Hausenblas
The main problem it's: the framework never was registered and the app is continuing re-deploying again and again. — enrique-carbonell

enrique-carbonell enrique-carbonell · Accepted Answer · 2015-09-28T14:44:25

The problem was solved changing environment variable CASSANDRA_SEED_COUNT, because by default CASSANDRA_SEED_COUNT = 2 and I was configured the app.json to deploy cassandra with only one node (CASSANDRA_NODE_COUNT=1).

If you are trying with all default config it's work fine with the environment variable, only you need be sure to has 3 or more slave nodes availables in the mesos cluster.

# The number of nodes in the cluster (default 3)
CASSANDRA_NODE_COUNT=3

# The number of seed nodes in the cluster (default 2)
# set this to 1, if you only want to spawn one node
CASSANDRA_SEED_COUNT=2

Deploy cassandra-mesos framework with marathon

1 Answers