Elastic Beanstalk Custom CloudWatch Logs

Question

I have an Elastic Beanstalk Environment on AWS where I deploy via Docker container. I've enabled the default logs from the UI Configuration of Beanstalk and we get standard logs such as eb-activity.log in CloudWatch.

My goal is to integrate the custom logfile: logger.log into CloudWatch.

Dockerrun.aws.json looks like this:

{
  "AWSEBDockerrunVersion": "1",
  "Ports": [
    {
      "ContainerPort": "3000"
    }
  ],
  "Logging": "/path/to/logs"
}

Then in .ebextensions/01_cloudwatch.config I have

files:
  /etc/awslogs/config/my_app_log.conf: 
    content: |
        [/var/log/eb-docker/containers/eb-current-app/logger.log]
        log_group_name=/aws/elasticbeanstalk/appname/var/log/eb-docker/containers/eb-current-app/logger.log
        log_stream_name={instance_id}
        file=/var/log/eb-docker/containers/eb-current-app/logger.log
    group: root
    mode: "000600"
    owner: root

The eb-user has full permissions of CloudWatch (CloudWatchFullAccess and CloudWatchLogsFullAccess). Also, I can validate the existence of /var/log/eb-docker/containers/eb-current-app/logger.log.

Still, logs don't appear in CloudWatch. In the same directory there are afc818177506-stdouterr.log files that appear in CloudWatch, they're being configured in /etc/awslogs/config/beanstalklogs.conf. Moreover, I made sure to do sudo service awslogs restart.

Any idea how to debug the problem or do you see any obvious mistake I did so logger.log does not appear in CloudWatch? Many thanks!

PS. I've already looked at the similar questions and have tried a good amount of time to solve the issue without success, so your help is needed.

Taterhead Taterhead · Accepted Answer · 2018-05-13T21:29:16

According to the Cloudwatch agent FAQ at the bottom of this reference, these are common reasons log events are not sent:

size of log event is > 256K
timestamp of log entries is off (>2 hours in future or >2 weeks in past) - perhaps clear your log file if you have a stale logger.log
log events older than the retention period
large log batches spanning more than 24hr

Lastly - to troubleshoot why cloudwatch log agent might be failing, check /var/log/awslogs.log and to troubleshoot the agent was installed correctly on the EC2 instance, check /var/log/awslogs-agent-setup.log

See if any of these tips help, and post back if you are still stuck with more information and we will continue to try and help you.

Elastic Beanstalk Custom CloudWatch Logs

1 Answers