Consul - Deploy different config for different hosts

2
votes

I am trying to deploy a consul cluster. I have the following machines:

consul-server01
consul-server02
consul-server03
web01
database01

I have 3 separate config files. One on each server respectively. 

/etc/consul.d/server/config.json
/etc/consul.d/web/config.json
/etc/consul.d/database/config.json

If I add a new server (say web02), how can I have it automatically adopt the web server config?

Does consul support configuration discovery, or do I need to use chef/puppet/ansible/salt to deploy the web config to the web server?

Resources: https://www.digitalocean.com/community/tutorials/how-to-configure-consul-in-a-production-environment-on-ubuntu-14-04

2
consul

2 Answers

3
votes

You can load your configurations into the initial Consul instances or clusters key/value store and then use consul-template to configure the additional nodes.

1
votes

Create a data-container deriving from consul, mount a named volume onto /data - called myconfig

create a small ruby/whatever script "generate_key.rb" which generates a key into /data/consul/encrypt.json if it yet does not exist. It ends up looking like this.

{ 'encrypt': 'somekey generated by consul keygen' }

For generating a key, use : consul keygen Start this script on container start ( ENTRYPOINT or CMD)

Setting up the consul-server

in the Dockerfile do

FROM consul
VOLUME /data/consul

# create a placeholder for the optional gossip key
RUN mkdir -p /data/consul && \
  echo "{}" > /data/consul/encrypt.json && \
  mkdir -p /consul/config &&
  ln -s /data/consul/encrypt.json /consul/config/encrypt.json

# you server config
COPY consul-config.json /consul/config/server_config.json

CMD ["agent","-server"]

Your conusl-config.json should look similar to this

 {
  "datacenter": "stable",
  "acl_datacenter": "stable",
  "data_dir": "/consul/data",
  "ui": true,
  "dns_config": {
    "allow_stale": false
  },
  "log_level": "INFO",
  "node_name": "consul",
  "client_addr" : "0.0.0.0",
  "server": true,
  "bootstrap":

 true
    }


# For every consul client
Create the same placeholder symlink 

    RUN mkdir -p /data/consul && \
      echo "{}" > /data/consul/encrypt.json && \
      mkdir -p /consul/config &&
      ln -s /data/consul/encrypt.json /consul/config/encrypt.json

why those symlinks and dummy files

This ensures, that if we now mount the data volume, the encrypt key get replaced by the one generated by the config - and if not, the server starts without it. Consul needs a proper json file, it is not allowed to be missing nor to be empty

docker-compose example

version: "2"
services:
  someconsuleclient:
    image: mymongodb
    container_name: someconsuleclient
    depends_on:
      - consul
    volumes_from:
      - dwconfig:ro
  consul:
    container_name: consul
    image: myconsulimage
    depends_on:
      - config
    volumes_from:
      - config:ro
  config:
    image: myconfigimage
    container_name: config
    volumes:
      - config:/data/
volumes:
  config:
    driver: local

So we have a config service to generate the encrypt.json, we have a consul server and we have a consul example client. Now you can add new consul-nodes very easy, while having a gossip encryption.

Of course, you can add arbitrary configuration, additional, for every client int /data/consul/custom_client.json in the bootstrap of your config container and share those across all clients. All .json files in the consul-config dir are merged, this way you can easily build "additions"