How can I stream a specific log file from multi-container Docker Elastic Beanstalk to CloudWatch?

Question

I have a web service deployed to an Elastic Beanstalk environment running the Docker Multi-Container stack. I have enabled Log Streaming to CloudWatch on the environment, so about five different log groups show up in Cloudwatch, and so when I click "Request Logs" from Beanstalk it loads a webpage that shows me all the log files, one after another. I've noticed that there are some logs on this web page that do not show up as Log Groups in CloudWatch, and these are the logs I really care about. My question is how do I get them to show up as CloudWatch Log Groups?

In particular, the five Log Groups that Elastic Beanstalk automatically created for me are:

/aws/elasticbeanstalk/my-web-service/var/log/docker-events.log
/aws/elasticbeanstalk/my-web-service/var/log/eb-activity.log
/aws/elasticbeanstalk/my-web-service/var/log/eb-ecs-mgr.log
/aws/elasticbeanstalk/my-web-service/var/log/ecs/ecs-agent.log
/aws/elasticbeanstalk/my-web-service/var/log/ecs/ecs-init.log

And when I look in the file that gets generated when I "request logs," those five are indeed there. But these other log files are also represented:

/aws/elasticbeanstalk/my-web-service/var/log/awslogs.log
/aws/elasticbeanstalk/my-web-service/var/log/docker
/aws/elasticbeanstalk/my-web-service/var/log/docker-ps.log
/aws/elasticbeanstalk/my-web-service/var/log/eb-commandprocessor.log
/aws/elasticbeanstalk/my-web-service/var/log/containers/my-svc-8edcf9cec583-stdouterr.log

It's that last one that I'm really interested in, the one ending in stdouterr.log. That's where my containerized application writes all of its log messages to. What I would like to see is a Log Group in CloudWatch that corresponds to that stdouterr.log file. As far as I can tell, the 12-digit ID that's in the log file name is the ID of the docker image that gets installed on the host, and is subject to change every time you restart the server. So I'm guessing I'll likely need to mount a volume, or something like that, in the Dockerrun.aws.json configuration? And furthermore I would guess that I'd then need to manually add a Log Group to CloudWatch? How can I get this file to show up?

Stephen Stephen · Accepted Answer · 2018-04-05T21:41:03

It looks like you currently only have the default logs being sent to Cloudwatch logs. You can add additional logs to the cloudwatch agent through your .ebextensions

### BEGIN .ebextensions/logs.config
option_settings:
  - namespace: aws:elasticbeanstalk:cloudwatch:logs
    option_name: StreamLogs
    value: true
  - namespace: aws:elasticbeanstalk:cloudwatch:logs
    option_name: DeleteOnTerminate
    value: false
  - namespace: aws:elasticbeanstalk:cloudwatch:logs
    option_name: RetentionInDays
    value: 7

files:
  "/etc/awslogs/config/stdout.conf":
    mode: "000755"
    owner: root
    group: root
    content: |
      [docker-stdout]
      log_group_name=/aws/elasticbeanstalk/environment_name/docker-stdout
      log_stream_name={instance_id}
      file=/var/log/eb-docker/containers/eb-current-app/*-stdouterr.log

commands:
  "00_restart_awslogs":
    command: service awslogs restart

### END .ebextensions/logs.config

source

How can I stream a specific log file from multi-container Docker Elastic Beanstalk to CloudWatch?

1 Answers