Mesos, Marathon, the cloud and 10 data centers - How to talk to each other?

5
votes

I've been looking into Mesos, Marathon and Chronos combo to host a large number of websites. In my head I should be able to type a few commands into my laptop, and wait about 30 minutes for the thing to build and deploy.

My only issue, is that my resources are scattered across multiple data centers, numerous cloud accounts, and about 6 on premises places. I see no reason why I can't control them all from my laptop -- (I have serious power and control issues when it comes to my hardware!)

I'm thinking that my best approach is to build the brains in the cloud, (zoo keeper and at least one master), and then add on the separate data centers, but I am yet to see any examples of a distributed cluster, where not all the nodes can talk to each other.

Can anyone recommend a way of doing this?

3
amazon-web-servicesdockermesosfigmarathon
Have you seen Docker machine? github.com/docker/machine Not quite what you're asking, but it allows you to control multiple Docker hosts from your laptop. - Adrian Mouat
I have seen Docker Machine, it doesn't really give me the autonomy that I'm looking for, but I believe that I may have to have multiple clusters... - Mister IT Guru

3 Answers

0
votes

I've got a setup like this, that i'd like to recommend:

  • Source code, deployment scripts and dockerfiles in GIT
  • Each webservice has its own directory and comes together with a dockerfile to containerize it
  • A build script (shell script running docker builds) builds all the docker containers, of which all images are pushed to a docker image repository
  • A ansible deploy deploys all the containers remotely to a set of VPSes. (You use your own deployment procedure, that fits mesos/marathon)
  • As part of the process, a activeMQ broker is deployed to the cloud (yep, in a container). While deploying, it supplies each node with the URL of the broker they need to connect to. In your setup you could instead use ZooKeeper or etcd for example.

I am also using jenkins to do automatic rebuilds and to run deploys whenever there has been GIT commits, but they can also be done manually.

Rebuilds are lightning fast, and deploys dont take much time either. I can replicate everything I have in my repository endlessly and have zero configuration.

To be able to do a new deploy, all I need is a set of VPSs with docker daemons, and some datastores for persistence. Im not sure if this is something that you can replace with mesos, but ansible will definitely be able to install a mesos cloud for you onto your hardware.

All logging is being done with logstash, to a central logging server.

0
votes

i have setup a 3 master, 5 slave, 1 gateway mesos/marathon/docker setup and documented here https://github.com/debianmaster/Notes/wiki/Mesos-marathon-Docker-cluster-setup-on-RHEL-7-with-three-master

this may help you in understanding the load balancing / scaling across different machines in your data center

1) masters can also be used as slaves 2) mesos haproxy bridge script can be used for service discovery of the newly created services in the cluster 3) gateway haproxy is updated every min with new services that are created

This documentation has 1) master/slave setup 2) setting up haproxy that automatically reloads 3) setting up dockers 4) example service program

0
votes

You should use Terraform to orchestrate your infrastructure as code.

Terraform has a lot of providers that allows you to manage different resources accross multiples clouds services and/or bare-metal resources such as vSphere.

You can start with the Getting Started Guide.