Is there a way to buffer X log messages from a CloudWatch log group and only then stream it to a lambda function? I'll elaborate:
I have an app that I registered it's CloudWatch logs to stream to a lambda function which formats the logs and pushes them to Elastic Search.
So the flow is the following: (app logs) -> (CloudWatch) -->(Lambda)-->(Elastic Search)
My problem is that my lambda function is invoked very often (most of the time single log message) and bombards ES with write requests, I would like to write the logs in bulks, i.e wait until 30 new logs and then invoke the lambda for the 30 logs bulk.
The only way I found to achieve this is to use Kinesis and Firehose but those services cost extra and I want to avoid this.
Are there any other alternatives to achieve this without using something like LogStash? I am assuming this is a very common usage so there must be some easy way to solve this.
Thanks,