SQS Messages not being removed

Question

I'm using python and when I read things from SQS they don't get removed properly and they get read again a few seconds later.

I use this code snippet to post a job

#!/usr/bin/python 

#simple code snippet that posts a message to SQS
import boto
from boto.sqs.message import Message
conn = boto.connect_sqs()
q = conn.create_queue('myqueue')
newJob.set_body("Message")
q.write(newJob)

Here is the code that digests messages

#!/usr/bin/python 

#simple code snippet that reads a message from SQS
import boto
from boto.sqs.message import Message
conn = boto.connect_sqs()
q = conn.get_queue('myqueue')

while True:
    try:
        print q.read(10).get_body()
    except:
        pass

After running both scripts the second one will print out 'Message' every 15 seconds or so.

I'm really confused as to why it's not being removed AND why it comes back only after 15 seconds. (It also doesn't show up in the AWS console, and messages I send from there are never processed)

Lei Shi Lei Shi · Accepted Answer · 2016-05-22T02:30:23

Disclaimer: I never used boto.sqs before and this answer is just based on reading its doc.

According to below description, it looks like a message after being read 1) is NOT automatically deleted , and 2) becomes invisible for a specific time period.

There is an optional parameter to create_queue called visibility_timeout. This basically controls how long a message will remain invisible to other queue readers once it has been read (see SQS documentation for more detailed explanation). If this is not explicitly specified the queue will be created with whatever default value SQS provides (currently 30 seconds).

update: Amazon SQS Doc confirms my above understanding.

...Therefore, Amazon SQS does not delete the message, and instead, your consuming component must delete the message from the queue after receiving and processing it.

...Therefore, Amazon SQS blocks them with a visibility timeout, which is a period of time during which Amazon SQS prevents other consuming components from receiving and processing that message...

So I guess you need to explicitly delete the message after you have successfully read it from the queue (if you don't want another client consume it).

SQS Messages not being removed

1 Answers