I am trying to configure AWS cloudwatch agent to send logs from my ec2 instances to Cloudwatch. I have 4 instances running under an ALB. The ec2 instances are each running a Java web service that can receive requests from multiple customers. Each customer's logs go to a separate log file on each instance- customerA.log, customerB.log etc (using logback's 'Sift' appender). Any customer request can go to any of the instances.
These customer names are not known in advance - the web service receives the customerId as part of the web request, and as new customers use the service there will be new log files.
I would like to be able to send logs to cloudwatch so they are collected and merged by customer name, so I can see a single log file per customer that is the pooled logs from all instances.
How can I do this? Using a configuration from the documentation I can see that wildcards can merge different log files into a single stream on CloudWatch, but I would like to replace {instance_id} with with some sort of dynamic placeholder for the exact name of the log file (which is named after customerId).
I'm not sure if this is even possible so if there are other configurations that can achieve the same goal that would be fine. Many thanks!
"logs_collected": {
"files": {
"collect_list": [
{
"file_path": "/media/logs-audit/customer*",
"log_group_name": "errors.log",
"log_stream_name": "{instance_id}"
},
]
}
}
