Workers Unable to Join Cluster, Allowed Role Set?

2
votes

I have obtained the latest DC/OS 1.8.7 CloudFormation template for 3 masters and 5 workers as described on their AWS installation page, and deployed it to my organization's AWS account.

My actual implementation cloud-init files are found here:

This setup works just fine, the workers discover the master and add themselves, I can schedule tasks onto the workers and things seem to just work. ZooKeeper is clustered, etc. Everything looks healthy, everything works, etc.

What is of note is that the only effective differences between the three types of instances are the roles at the bottom of the files:

masters:

  - path: /etc/mesosphere/roles/master
    content: ""

  - path: /etc/mesosphere/roles/aws_master
    content: ""

  - path: /etc/mesosphere/roles/aws
    content: ""

public workers:

  - path: /etc/mesosphere/roles/slave_public
    content: ""

  - path: /etc/mesosphere/roles/aws
    content: ""

services workers:

  - path: /etc/mesosphere/roles/slave
    content: ""

  - path: /etc/mesosphere/roles/aws
    content: ""

If I change any of these roles to anything but slave or slave_public, these instances will never be able to join the DC/OS cluster. Since my requirements here dictate that I need at least four types of roles, this is somewhat of a roadblock.

I need roles for the following:

  • ☑ master
  • ☑ public worker
  • ☐ services worker
  • ☐ data worker

Due to my network layout and organizational needs, these extra types aren't negotiable.

Are there some extra steps required to have DC/OS allow additional worker roles?

1
amazon-web-servicesmesosdcos

1 Answers

3
votes

The master, slave, and slave_public arguments are the only ones known by the DC/OS installer.

While slave_public argument does statically allocate all node resources to the slave_public role, the slave argument actually allocates all node resources to the * role, which is used for tasks with no role specified.

In order to assign resources to other roles, there are two methods:

  1. Statically allocate the resources to custom roles by modifying the json content of MESOS_RESOURCES in the /var/lib/dcos/mesos-resources file on each agent node.
  2. Dynamically allocate the resources to custom roles using the Dynamic Reservation Mesos API: http://mesos.apache.org/documentation/latest/reservation/