Sharing data in distributed system environment

0
votes

I have a question regarding sharing data in distributed system environemnt. Usually I am using scenario A, ie. I am pushing messages from system A to a remote Queue such as Amazon SNS/SQS and I am expecting system B to pick the message up when system B is up and running.

I have thought about it and I came with an alternative that I am not sure is good enough. Basically, System A now pushes message to local queue and local queue pushes message to system A. Prerequisites for this are that both systems A and B are up and running and that queue is specific only for system A, ie. there is no system C, D, E... that are pushing message to the same (duplicate) local queue somewhere on C, D, E.

Is this a good alternative?

I was also considering scenario B as a backup plan for scenario A, when remote queue is unavailable and I dont want to lose a message. In that case system B would be a remote queue.

enter image description here

enter image description here

1
eventsmicroservicesdistributed-system

1 Answers

1
votes

Well the scenario C is actually the way distributed messaging should be and is actually done . This is called outbox pattern in Async communication. There kind of patterns are used for resiliently publishing to the event bus . Its very much like mail box where your mail box keeps your email in the outbox until delivered. The bit of a tweak is that instead of local queue use persistent storage.

But you need to have worker which will be looking for pending messages and will publish them as soon as queue is available.

You can find more about this pattern here.

Outbox pattern

Resiliently publishing to the event bus