How to decide whether Kafka is suitable for event storing?

Question

I consider using Kafka for an event sourcing project (there are agents who produces events which will be passed through Kafka to specific processors...)

As I have no practival experience with event sourcing and Kafka I've done some research online. And I've found out that Kafka is not good for event sourcing (Using Kafka as a (CQRS) Eventstore. Good idea?). There is a quote:

SQL/NoSQL store and Kafka as broker is better choice than leaving Kafka handle both roles to create complete feature full solution

There is a Kafka co-author telling

Kafka will work very well as a log for event sourcing. It is fault-tolerant, scales to enormous data sizes, and has a built in partitioning model.

I still can not consider whether using Kafka is a good idea or not in my case. Do I need an event store even if Kafka has topics?

I would like to have a simple list of questions I should ask myself which helps me to decide whether use Kafka alone or use Kafka + event store.

Answer 1

How to decide whether Kafka is suitable for event storing?

First, you need to decide what you think "event sourcing" means.

You might want to review What do you mean by "Event-Driven", by Martin Fowler.

One thing to think about is whether or not there is a domain invariant that needs to be maintained on a partition of events. In most dedicated event stores, there are mechanisms available to ensure that new events are appended to the history only if they are consistent with the previously written events.

In many event sourced solutions, you want many short histories - a history for each order fulfillment, for each shipment, for each instance of a process - so that you can support concurrent writes to different instances without losing the consistency property.

If you need support for these, then you need to look into how well Kafka's feature set aligns with them (no different from evaluating any other off-the-shelf solution, really).