How to deliver AWS SNS message to all instances of particular micro service

1
votes

I've got rather rare requirement to deliver SNS topic message to all micro service instances.
Basically it's kind of notification that related data had changed
and all micro service instances should reload their internals from data source.

We are using TerraForm to create our infrastructure, with Kong api gateway.
Micro Service instances could be created 'on the fly' as system load is increased, so subscriptions to topic could not be created in TerraForm stage.
Micro Service is standard SpringBoot app.

My first approach is:

  • micro service is exposing http endpoint that can be subscribed to SNS topic
  • micro service on start will subscribe itself (above endpoint) to required SNS topic, unsubscribe on service shutdown.

My problem is to determine individual micro service instances urls, that can be used in subscription process.

Alternative approach would be to use SQS, create SQS queue per micro srv instance (subscribe it to sns).

Maybe I'm doing it wrong on conceptual level ?
Maybe different architecture approach is required ?

1
amazon-web-servicesmessagingamazon-sns
If you use sns subscriptions, once your services scale down, you end up with a number of dead subscription endpoints. Same for sqs. If the services change, sqs queues will get abandoned. What about some sort of a message broker? For instance, a lambda triggered by SNS would query your ecs services, and iteratively push a message to all of them? – Marcin

1 Answers

1
votes

It might be easier for the microservices to check an object in Amazon S3 to "pull" the configuration updates (or at least call HeadObject to check if the configuration has changed) rather than trying to "push" the configuration update to all servers.

Or, use AWS Systems Manager Parameter Store and have the servers cache the credentials for a period (eg 5 minutes) so they aren't always checking the configuration.