Try “Automated software delivery using Docker Compose and Amazon ECS”, but fail at Compose2Cloudformation

Question

■Disability Summary

Try "Automated software delivery using Docker Compose and Amazon ECS", but fail at Compose2Cloudformation at the end of the CodePipeline.

■Verification environment

OS：Windows 10 Professional Terminal：MINGW64 AWS CL：aws-cli/2.2.13 Python/3.8.8 Windows/10 exe/AMD64 prompt/off Docker Compose：Docker Compose version 1.0.17

■Procedures used for reference

https://aws.amazon.com/jp/blogs/containers/automated-software-delivery-using-docker-compose-and-amazon-ecs/

Translated from Japanese (contents are the same as the above link) https://aws.amazon.com/jp/blogs/news/automated-software-delivery-using-docker-compose-and-amazon-ecs/

■Target Demo Project

https://github.com/aws-containers/demo-app-for-docker-compose.git

docker-compose.yml

x-aws-vpc: ${AWS_VPC}
x-aws-cluster: ${AWS_ECS_CLUSTER}
x-aws-loadbalancer: ${AWS_ELB}

services:
  frontend:
    image: ${IMAGE_URI:-frontend}:${IMAGE_TAG:-latest}
    build: ./frontend
    environment:
      REDIS_URL: "backend"
    networks: 
      - demoapp
    ports:
      - 80:80

  backend:
    image: public.ecr.aws/bitnami/redis:6.2
    environment:
      ALLOW_EMPTY_PASSWORD: "yes"
    volumes:
      - redisdata:/data
    networks:
      - demoapp

volumes:
  redisdata:

networks:
  demoapp:

■error log

compose-pipeline-ExtractBuild:17ef28f6-b566-47ed-a96d-0bb7a34cd47f

[Container] 2021/06/29 09:15:25 Running command docker context create ecs demoecs --from-env
Successfully created ecs context "demoecs"

[Container] 2021/06/29 09:15:25 Running command docker context use demoecs
demoecs

[Container] 2021/06/29 09:15:25 Phase complete: PRE_BUILD State: SUCCEEDED
[Container] 2021/06/29 09:15:25 Phase context status code:  Message: 
[Container] 2021/06/29 09:15:25 Entering phase BUILD
[Container] 2021/06/29 09:15:25 Running command echo Convert Compose File
Convert Compose File

[Container] 2021/06/29 09:15:25 Running command docker --debug compose convert > cloudformation.yml
level=debug msg=resolving host=098456798948.dkr.ecr.ap-northeast-1.amazonaws.com
.
.
.
level=debug msg="searching for existing filesystem as volume \"redisdata\""
multiple filesystems are tags as project="src", volume="redisdata"

[Container] 2021/06/29 09:15:26 Command did not exit successfully docker --debug compose convert > cloudformation.yml exit status 1
[Container] 2021/06/29 09:15:26 Phase complete: BUILD State: FAILED
[Container] 2021/06/29 09:15:26 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: docker --debug compose convert > cloudformation.yml. Reason: exit status 1
[Container] 2021/06/29 09:15:26 Entering phase POST_BUILD
[Container] 2021/06/29 09:15:26 Phase complete: POST_BUILD State: SUCCEEDED

I tried manually adding the following Policy to "compose-pipeline-ExtractBuildRole-1CB5P2KEHAL6Q" and re-running it, but the situation remained the same. "AmazonElasticFileSystemFullAccess" — Hidetoshi Maekawa

mreferre mreferre · Accepted Answer · 2021-06-29T15:20:41

Co-author of the blog here (thanks for giving it a try). So the message is kind of interesting:

searching for existing filesystem as volume \"redisdata\""
multiple filesystems are tags as project="src", volume="redisdata"

It almost feels like it's trying to find an existing EFS for this application to re-use (and if it doesn't exist it will create it) but it says it finds "multiple" (which should not happen because either it doesn't exist and it will be created or one exists and it will be re-used). Can you check if by any chances you see 2 or more EFS file systems with the tags project="src" and volume="redisdata"?

Also, at which point of the tutorial are you hitting this problem? At first deployment? Or when you update the application and re-deploy?

Anyway, as we were digging into this we found there was a missing action in the IAM policy that prevented the pipeline to properly interact with EFS. We just updated the repo with the missing action.

We believe these two things (the missing action and the error message you received are not strictly related) but can I ask you to remove everything and restart from scratch? Please make sure you delete manually the EFS volumes (because docker does not delete them) and also that you follow the Clean UP section at the end of the blog to delete everything properly?

Sorry for the inconvenience.

Try “Automated software delivery using Docker Compose and Amazon ECS”, but fail at Compose2Cloudformation

2 Answers