I have a Lambda function that has a Python handler that submits a job to AWS Batch via boto3
client:
client = boto3.client('batch', 'us-east-1')
def handle_load(event, context):
hasher = hashlib.sha1()
hasher.update(str(time.time()).encode())
job_name = f"job-{hasher.hexdigest()[:10]}"
job_queue = os.environ.get("job_queue")
job_definition = os.environ.get("job_definition")
logger.info(f"Submitting job named '{job_name}' to queue '{job_queue}' "
f"with definition '{job_definition}'")
response = client.submit_job(
jobName=job_name,
jobQueue=job_queue,
jobDefinition=job_definition,
)
logger.info(f"Submission successful, job ID: {response['jobId']}")
I can see this Lambda function submit the Batch job in CloudWatch logs but it always times out before the response comes back. I never see these jobs show up in the queue, so I'm not sure where things go after they are submitted, it seems that the Lambda is always timing out before the response comes back, I have little else to go on.
I have successfully added a job to the queue via AWS CLI, using the same queue and definition ARNs that are used in the Lambda's Python code. This job can be seen in the queue under the runnable tab (presumably the job will be started at some point in the near future).
The job submission with AWS CLI comes back instantly, so there must be something amiss on the Lambda configuration preventing the job submission. Perhaps I'm not using the correct role for the Lambda that submits the job, or have some other permissions that are amiss causing the timeout? The Lambda has permission for the batch:SubmitJob
action allowed on all resources.
client.submit_job()
is timing out? Is your lambda in VPC? – Marcin