Mesos / Marathon Roles and Applications

Question

I want to run a Mesos/Marathon cluster with a mix of public and private apps. It seems like it should be possible to leverage roles for this, but I'm not having success.

As a proof-of-concept, I have a Mesos cluster with public and private roles:

# curl --silent localhost:5050/roles | jq '.' | grep name
"name": "*",
"name": "public",
"name": "private",

The problem arises with Marathon and my Marathon apps. I'm unable to set default_accepted_resource_roles to a value that's different from mesos_role. For example, if I set --mesos_role=public and --default_accepted_resource_roles=public, I'm able to run apps, but only if they include "acceptedResourceRoles": [ "public" ] in the application description. Conversely, I can set everything to private and run private apps. (Although, ironically, if I set everything to * I'm unable to run my apps. I don't understand that at all.)

Ideally, I'd like to run this way:

mesos_role=*
default_accepted_resource_roles=private

I would expect this to give Marathon access to all my roles but start apps with the private role unless they specifically request the public role. Unfortunately, this results in this error:

Exception in thread "main" java.lang.IllegalArgumentException: requirement failed: --default_accepted_resource_roles contains roles for which we will not receive offers: private

and Marathon refuses to start.

Is it possible to run Marathon with --mesos_role=* and --default_accepted_resource_roles=private? If not, I don't understand why the roles exist. But I'd rather run as much as possible in a single cluster rather than spinning up separate clusters for public and private apps.

@janisz Thanks. I'll watch that issue. There seems to be some confusion on their side, too. From the ticket: I'd like Marathon to be able to launch different services under different roles. I believe they were suggesting that this task needed to be accomplished before they could do that. I still only want Marathon to run with one role (*), but I want it to be able to accept offers from other roles. It seems like that was the original design goal for the roles, at least from the Mesos side. — aayore

varvind varvind · Accepted Answer · 2016-02-17T04:30:25

@aayore - as a workaround we deploy multiple marathon. main-marathon and couple of others marathons which run on the main marathon as app, typically one per role.

Say you have 2 roles "kafka" and "hdfs" e.g. main marathon runs with default role (which is *) marathon 2 runs on main-marathon with --mesos_role kafka (command line option) marathon 3 also runs on main-marathon with --mesos_role hdfs

for an app to use resources allocated to kafka role, you need to launch the job in kafka-marathon instead of the main marathon.

Mesos / Marathon Roles and Applications

1 Answers