How to decouple messaging middleware from microservices?

0
votes

We have a bunch of microservices written in different programming languages by a variety of teams spanning across different companies. All these services communicate with each other over RabbitMQ(AMQP) middleware. There are consumers and publishers in each microservice which listen to incoming messages and publish on a RabbitMQ exchange respectively. The current architecture is shown in the image below

enter image description here

I want to decouple the RabbitMQ specific consumers and publishers from the microservices and implement them in a separate shared library as follows.

enter image description here

This shared library shall implement RabbitMQ exchange bindings. The consumer in this shared library, should listen to any incoming messages and invoke the relevant method in the target language. Same for publishers, the target language should be able to invoke a method in the shared library responsible to publish message on RabbitMQ.

The reason I want to have a shared library is

  1. Since my current services directly implement RabbitMQ bindings, the integration test cases for each of the services also have a dependency on RabbitMQ.
  2. I can easily swap RabbitMQ for any other messaging service(for example Kafka) without changing my services, as this makes the services oblivious of the underlining messaging service.

The problem that I am currently facing is creating language bindings.

If I decide to write the shared library in Java, since most of the message queuing services have good support for Java, how to achieve language bindings?

I am still planning this and haven't started with the implementation. Please provide some suggestions on approaching this problem.

1
architecturerabbitmqmicroservicesmessage-queue
Are you sure this is possible? Different languages have different models of execution (interpreted vs compiled, etc). I imagine it would be really challenging to make this library compatible with all the languages. Can't you make a version of the shared library for each language? - Cosmin Ioniță
@CosminIoniță The services that I have menioned are developed by different teams spannign across different companies. Creating a wrapper for each service would increase the possibility of potential bugs. No to mention writting additional test cases(unit, integration, application). This shared library approach is an attempt to tie all the services together wihtout major changes to the existing services. - Nishant123
Yeah, don't do this. - Software Engineer
@SoftwareEngineer I would appretiate if you could share an alternative approach to this problem. - Nishant123
There isn't one. You're already doing it the easiest way. You can swap rabbit for any other amqp broker and your app will continue to function. You have chosen an amqp abstraction - messaging. You cannot write an abstraction that allows you to change from rabbit to Kafka because they're fundamentally different paradigms and require your application to be rearchitected. Nobody has done this well (though there is an amq connector to bridge rabbit to Kafka, this is used for a different purpose and will not allow your clients to connect to Kafka through amqp) - Software Engineer

1 Answers

1
votes

you are not actually decopling them, you are moving the coupling to a lib in a difficult way, not easy to maintain. Try to follow KISS principle