Push Messaging with Amazon SNS, SQS and PHP

2
votes

I want to use Amazon SNS to send time-critical mobile PUSH notifications to 20 million devices. Each topic can have up to 10,000 devices, and I can create up to 3,000 topics. Using the Amazon PHP SDK would mean sending 2000 API calls at 1 second each - 33 minutes in total. This is no good for time-critical messages.

I have created an SQS queue and subscribed that to the SNS topic. When I send my PUSH message to the SQS queue, it doesn't get delivered - it remains in the queue.

How can I use these services to send the messages more quickly?

Thank you!

4
amazon-web-servicespush-notificationamazon-sqsgoogle-cloud-messagingamazon-sns
I think you have the connection reversed. When you subscribe a SQS queue to an SNS topic, the queue will receive messages published to the topic, but not the other way around. See the documentation for more information. – Bob Kinney
I think you're right. So, how would I structure a system to send PUSH notifications to this volume of users? – SteveMc
You may want to look into using SNS->SQS and using multiple workers to consume the queues to push a higher throughput if you are unable to get the desired throughput from a single php thread. What are you basing the "1 second each" measure for operations with the AWS SDK for PHP? – Bob Kinney
I was basing that on some preliminary tests, showing CURL requests taking 0.5-1 second. Regardless, the bottle-neck is sending the requests to the SNS topics, not having SNS process the PUSH notifications. As a result, I don't think SQS will actually be of any benefit in my circumstance - the 'worker' is actually the SNS service as it is sending the PUSH message. I think a better solution is to use PHP Multicurl to batch send my message to the maximum 3,000 topics. Thank you Bob and @felipe-garcia for your help and advice. – SteveMc
@SteveMc, using multiple threads you should be able to do hundreds of publishes per second from a single machine without too much difficulty – tster

4 Answers

0
votes

Did you check the Queue Policy? When you create it, by default it has no policy and only the owner is able to use it.

Check the IAM user you are using with the API and make sure you have the policy on the SQS granting rights to him.

I have used SNS in the past to deliver around 1.5MM notifications each month and I had no problems. But I have no proven track record or experience at the rate you need. But I suggest you to take a look at the following article http://www.quora.com/Push-Notifications/Which-is-best-to-use-Amazon-SNS-Google-Cloud-Messaging-or-Parse-Why which talks a little bit about SNS x GCM.

Hope this helps.

0
votes

Since you don't specify the complete workflow of the notification I would be guessing, and my guess would be that you have a wrong approach.

For instance, if you need to send 100 push notification and for that you create a 100 SNS requests from your app at once, then you are adding an unnecessary layer of complexity. Just send the 100 push from your app and skip SNS.

The right way to do it is to fire a single SNS action that will hit all the workers and they will do the job on demand as they get the "order".

Hope that helps.

0
votes

10,000 devices per topic * 3,000 topics would allow you to send a message to 30 million devices. Why not just duplicate your topic across all 3,000 topics and publish your message to each topic? Users would get subscribed to one of those 3,000 topics. I haven't actually tried this, but seems to be the most straight-forward approach.

0
votes

Use Guzzle to send the requests in parallel.

$client->send(array(
    $client->get('http://www.example.com/foo'),
    $client->get('http://www.example.com/baz'),
    $client->get('http://www.example.com/bar')
));