I've been looking into Apache Camel and Kafka over the past two days in hopes of learning about messaging frameworks/brokers. Is a possible use case of Camel/Kafka using Kafka as the message broker while implementing the producers and consumers with Camel? I saw a brief example of something similar, but can't seem to find it again. If not, what is the point of the Camel:Kafka component?
1
votes
Have you looked here camel.apache.org/kafka.html? Has examples of consuming and producing using Camel.
– Erik Pearson
I have. Probably should have been more direct with my question. Is it that easy using Camel? I see going through the documentation of Kafka that the logic behind getting producers up and running with proper partitions alongside the consumers handling partitions with the logic behind rewinding can get pretty nasty. Is it really as easy as the examples they provide? Sorry if my questions are loaded, but the documentation behind the Kafka component isn't too informative to me.
– Clannadqs
Can you tell me what benefit will apache camel provide over writing our own producer and consumers?
– Abhimanyu
Not sure if I'm the best one to be answering this, but I've done more research since when I asked this question. Camel adds more abstraction to handling middleware. Meaning you can focus less on how Kafka works. Pretty sure most systems could care less about the finer levels of middleware as long as it works as they intend it to. Also, it allows you to switch middleware out without too much hassle. You're basically hiding your middleware.
– Clannadqs
Start with understanding your delivery guarantee requirements - do you need exactly once, at least once, or at most once?
– Chris Snow
1 Answers
5
votes
Yes Apache Camel makes using Kafka easier as it hides a bunch of the complexities, which is the main point about Camel components. However if you need to do something really advanced or be in control yourself then sometimes a Camel component may lack a functionality for that, or some hooks/apis you need, and if so people ask for that and we improve these components over time, to the communities requirements. And if you cannot wait/do that, then you do NOT have to use a Camel component and can use the Kafka API yourself directly - after all its all just Java.
There is also an Camel example here: https://github.com/apache/camel/tree/master/examples/camel-example-kafka.
And the Camel in Action 2nd edition book covers Camel with Kafka in its cluster chapter.
