Consul configuration in docker-compose

Question

I'm trying to setup a consul cluster locally using docker-compose, and my source of inspiration (this is my first interaction with consul), is this consul.yml. This configuration contains some syntax that I'm seeing for the first time.

I'm figuring that this

consul-agent-1: &consul-agent
    image: consul:latest

serves as some kind of base (like in inheritance) ?

and this part is the derived ? I don't know how to call it

consul-server-bootstrap:
    <<: *consul-agent

Please correct me if I'm wrong about this assumption.

But what I find very strange is this part in consul-agent-1

command: "agent -retry-join consul-server-bootstrap -client 0.0.0.0"

It does 'something' (I'm not sure what exactly) with consul-server-bootstrap service.

I don't understand it, I mean if I'm correct that consul-agent-1 serves as some base template for other services, and if consul-server-bootstrap uses the template, isn't this a circular reference ?

Answer 1

This is using a feature of YAML called anchors and merge keys to template / re-use some of the configuration in the compose file. See this StackOverflow answer for more info.

Meaning of ampersand (&) in docker-compose.yml file

Answer 2

Yes, &consul-agent and <<: *consul-agent are features of YAML files that allow to re-use some parts by merging them with "base dictionary" (as mentioned by Blake). Here are a couple of additional articles that showcase DRY for YAML files, one and two.

However, in this part

command: "agent -retry-join consul-server-bootstrap -client 0.0.0.0"

consul-server-bootstrap has nothing to do with YAML anchors that used to "inherit/merge directories". Instead, it is related to some form of service discovery intrinsic to docker compose, i.e. each container gets assigned certain hostname that can be referenced in other parts of docker compose files or within each service configuration for communication between services/containers.

When you run

docker-compose up

there will be 3 containers that run consul agent in "client" mode and 3 containers that run it in "server" mode. Consul agents that run on a backend (or inside a container) in "client" mode report to consul servers.

A server node takes on the additional responsibility of being part of the consensus quorum. These nodes take part in Raft and provide strong consistency and availability in the case of failure

So consul-server-1, consul-server-2 and consul-server-bootstrap carry the same functions, where the latter just waits until the specified number of servers are available, i.e. -bootstrap-expect 3, and then bootstraps the consul cluster.

See official docs for more information about consul agent and its' configuration.

TLDR;

Note, that without YAML anchors them here is how demo setup of Consul with docker-compose.yml would look like:

version: '3'
)
services:
  consul-agent-1:
    image: consul:latest
    networks:
      - consul-demo
    command: "agent -retry-join consul-server-bootstrap -client 0.0.0.0"

  consul-agent-2:
    # Complete copy of 'consul-agent-1'
    image: consul:latest
    networks:
      - consul-demo
    command: "agent -retry-join consul-server-bootstrap -client 0.0.0.0"

  consul-agent-3:
    # Complete copy of 'consul-agent-1'
    image: consul:latest
    networks:
      - consul-demo
    command: "agent -retry-join consul-server-bootstrap -client 0.0.0.0"

  consul-server-1:
    # Uses only image and networks from 'consul-agent-1'
    # while 'command' is overwritten
    image: consul:latest
    networks:
      - consul-demo
    command: "agent -server -retry-join consul-server-bootstrap -client 0.0.0.0"

  consul-server-2:
    # Complete copy of 'consul-server-1'
    image: consul:latest
    networks:
      - consul-demo
    command: "agent -server -retry-join consul-server-bootstrap -client 0.0.0.0"

  consul-server-bootstrap:
    # Uses only image and networks from 'consul-agent-1'
    # while 'command' is overwritten and 'ports' are defined
    image: consul:latest
    networks:
      - consul-demo
    ports:
      - "8400:8400"
      - "8500:8500"
      - "8600:8600"
      - "8600:8600/udp"
    command: "agent -server -bootstrap-expect 3 -ui -client 0.0.0.0"

networks:
  consul-demo: