Kafka vs JMS for event publishing

0
votes

In our scenario we have a set of micro services which interact with other services by sending event messages. We anticipate millions of messages per day at the peak. Every message is targeted to one or more listener types. Our requirements are as follows:

  1. Zero lost messages.
  2. Ability to dynamically register multiple listeners of a specific type in order to increase throughput.
  3. Listeners are not guaranteed to be alive when messages are dispatched.

We consider 2 options:

  1. Send each message to JMS main queue then listeners of that queue will route the messages to specific queues according to message content, and then target services will listen to those specific queues.
  2. Send messages to a Kafka topic by message type then target services will subscribe to the relevant topic and consume the messages.

What are the cons and pros for using either JMS or Kafka for that purpose?

2
apache-kafkajmsmicroservicesevent-driven
If the Kafka producer can differentiate message types and send particular message types to particular topics then why can't the JMS producer do the same thing to avoid the intermediate step of having listeners to move the messages? - Justin Bertram
If JMS producer sends to topics, messages will be lost if the consumer is not alive. - Hanan Tomer
But you stated that you're considering sending the message to a JMS queue, not a topic. After reading your updated question it's not clear that you want to use publish-subscribe semantics (i.e JMS topics) at all. - Justin Bertram

2 Answers

2
votes

Your first requirement is "zero lost messages". However, if you want publish-subscribe semantics (i.e. topics in JMS), but listeners are not guaranteed to be alive when messages are dispatched then JMS is a non-starter as those messages will simply be discarded (i.e. lost).

0
votes

I would suggest to go with Kafka as it has fault tolerance mechanism and even if some message lost or not captured by any listener you can easily retrieve it from Kafka cluster.

Along with this you can easily add new listener / listener in group and kafka along with zookeeper will take care of managing it very well.

In summary, Kafka is a distributed publish-subscribe messaging system that is designed to be fast, scalable, and durable. Like many publish-subscribe messaging systems, Kafkamaintains feeds of messages in topics. Producers write data to topics and consumers read from topics.

Very easy for integration.