Make permissions for SQS FIFO queue based on Message Group ID?

1
votes

I'm building a system which sends commands to external systems (all identical but with different locations and IDs).

The commands are sent to a FIFO SQS queue and the external systems read and delete from that queue.

Currently the plan is to create one queue for each external system, so I'd just have a Lambda that updates the list of queues when the DB table of systems is changed.

But I can see that the SQS FIFO supports message group IDs so I wonder if I should just have one single queue, where all systems only read from their own message group ID.

I like the simplicity of this solution - however, I cannot see a way to limit access for reading and deleting messages for a specific message group, which means that if one external system is compromised, its credentials can be used to hijack the shared queue for all external systems and therefore, take down everything.

Is there a workaround for this, so I can set some permissions for a specific queue and message group ID, in any way?

I am also concerned about the missing option of purging only one group of messages, not the entire queue.

1
amazon-sqsamazon-iam

1 Answers

4
votes

You can't read "from" a specific message group in a FIFO Queue, and there are no related permissions.

Message groups are opaque labels that tell the FIFO queue whether any two messages must be delivered to consumers in strict FIFO order relative to each other. If two messages share the same message group, they must be strictly ordered, but two messages with different message-group-ids do not need to be strictly ordered.

This capability allows faster overall processing of messages when there are parallel identical consumers, because without this feature, only one consumer could be handling messages at any one time, and the overall throughput of the queue would be limited to how quickly a single consumer could handle a message (since no messages would be delivered to another consumer as long as a single message was in flight).

Message Group ID

The tag that specifies that a message belongs to a specific message group. Messages that belong to the same message group are always processed one by one, in a strict order relative to the message group (however, messages that belong to different message groups might be processed out of order).

[...]

If you require a single group of ordered messages, provide the same message group ID for messages sent to the FIFO queue.

https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/FIFO-queues.html

Note also that while the ReceiveMessage API allows you to ask that the message-group-id be returned with each message, it has no provision for specifying which message-group-id you want to receive messages from, because that isn't the purpose of this feature.