SQS fifo trigger invoke Lambda Function (1 message - 1 invocation)

Question

I have a SQS FIFO queue triggering a Lambda function. I sent 10 messages (all different) and the lambda was invoked just once.

Details:

SQS
- Visibility timeout: 30 min
- Delivery delay: 0 secs
- Receive Message Wait Time: 0 secs
Lambda:
- Batch size: 1
- timeout: 3secs

I don't see any errors on Lambda invocations.

I don't want to touch the delivery delay, but if I increase, seems working. The avg duration time is less than 1,5ms

Any ideas how I can achieve this? Should I increase the delivery delay or time out?

The message is being sent from a ecs task with the following code:

from flask import Flask, request, redirect, url_for, send_from_directory, jsonify
app = Flask(__name__)
from werkzeug.utils import secure_filename
import os
import random
import boto3

s3  = boto3.client('s3')
sqs = boto3.client('sqs',region_name='eu-west-1')

@app.route('/', methods=['GET'])
def hello_world():
  return 'Hello World!'

@app.route('/upload', methods=['POST'])
def upload():
  print (str(random.randint(0,9)))
  file = request.files['file']
  if file:
    filename = secure_filename(file.filename)
    file.save(filename)
    s3.upload_file(
        Bucket = os.environ['bucket'],
        Filename=filename,
        Key = filename
    )
    resp = sqs.send_message(
        QueueUrl=os.environ['queue'],
        MessageBody=filename,
        MessageGroupId=filename
    )
    return jsonify({
        'msg': "OK"
    })
else:
    return jsonify({
        'msg': "NOT OK"
    })

Are you uploading the same file each time? If yes, @Azize has the answer for your question. — Maurice

Azize Azize · Accepted Answer · 2021-04-27T20:10:24

Check if this helps:

The message deduplication ID is the token used for deduplication of sent messages. If a message with a particular message deduplication ID is sent successfully, any messages sent with the same message deduplication ID are accepted successfully but aren't delivered during the 5-minute deduplication interval.

https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/using-messagededuplicationid-property.html

At least it explains why it works when you increase delivery delay.

SQS fifo trigger invoke Lambda Function (1 message - 1 invocation)

1 Answers