Can not reach Amazon SES sending limit because of throttling errors

3
votes

we send messages with Amazon SES using SES API. Our send rate is now 90 messages per second. But we are getting throttling exception even when we don't reach this limit but just trying to approach it.

Right now we can steadily send at the rate of 30 messages per second. The question is how to send faster.

Let me dive into some more details and clarify the question.

  • It might take from 0.3 to 3 seconds for a single API send request to complete. This is why if we would send messages sequentially we would hardly get 1 message per second speed.
  • Luckily we can send messages in parallel and this is what we are doing.
  • For each thread we check that it does not send more then allowed number of messages per second. For example if we have 40 threads then we don't allow each thread to send more then 2 messages per second. Yes, this is not optimal.
  • We register every message send and the time when the API request has completed (when we got the response from API). This allows to get some statistics.
  • When we restrict the sending limit to be less then allowed limit (like 60 instead of 90) everything works fine.
  • When we try to send at maximum limit we start getting throttling errors. Like when have speed of 80 requests per second reached we start getting exceptions.

This allows me to put the questions:

Q: How to send messages with highest allowed speed?

Let's start with another question - 'How does SES calculate the number of messages to check send rate?'

Let me guess. When we submit a new request they look at the number of requests submitted during last second from current moment and if this number is less then our limit the request is accepted.

But wait. If we have 40 threads and each thread can not send more then 2 messages per second then we can never reach the limit. But we do get the exceptions.

Research

There is a great blog post on Amazon SES blog about handling the limits. We are trying to adopt this approach but have not succeed yet.

We are using PHP in our application and PHP SES SDK.

I guess this is quite common task but for some reason I'm not lucky to find the complete solution.

Any help or ideas would be greatly appreciated. Thank you.

1
phpamazon-sesthrottling

1 Answers

1
votes

The key take away is:

A variety of factors can affect your send rate, e.g. message size, network performance or Amazon SES availability.

Based of what you've said it seems like you're using a bit of fuzzy logic to try and calculate how many messages you're sending. This won't be perfect so if your AWS limit is 90p/s then getting setting your code lower, e/g to 60p/s makes sense (once again this all depends on how accurate your estimates are).

You should consider the other approaches as you've mentioned, such as "Exponential backoff" as described in the link you provided.

Another thing you can consider is taking advantage of a queue, like SQS. This way you can pick tasks off the list as quick as possible, and if you're a little too quick you can always back off and then jump back on the queue as soon as possible.