I currently have a lambda function which stops my EC2 instances (4 virtual servers) and have created a schedule using Cloudwatch to kick off this lambda function to stop my servers at 6:30PM Mon-Fri. That part works beautifully.
However, if I run an application specific analysis batch on my servers at, for example, 3PM, some times the application running the analysis takes 4-5 hours to complete, which means that CloudWatch would essentially stop the servers at 6:30PM in the middle of the analysis process. Is there a way to build in an exception to check if CPU utilization on the servers is greater than 10%, to delay the cloudwatch event?
I tried creating an alarm event in CloudWatch to shutdown the specific EC2 servers if CPU utilization metric is <10% but that would essentially turn them off during the work day if not being utilized, and that is not what I need.
Ultimately, I'd like to be able to write a script (lambda function) that would check to see if CPU utilization at 6:30PM Mon to Fri is below 10%, if it is, then stop the servers. If not, then keep checking every hour. At this point, I'd love any and all suggestions please. Thank you in advance for your help!