I'm planning a model for microservices using event sourcing. To achieve a high scalability and high throughput handling capacity, I'll use Kafka as a message broker for the microservices.
At this point, I have questions about the implementation of the model to be able to have the benefits of Kafka topic and partition. There are some requirements that my model needs to fit:
- Microservices must ingest data from the message broker (POST/PATCH/PUT/DELETE)
- Data consistency is mandatory, if entity A needs previous existence of entity B, then must exists only records of entity A that points to valid records of entity B
- The microservices can't couple their domains (refering to DDD)
- The system must handle high thrgoughput of read and writes operations
Well, with this requirements in mind, I've endend up with a model that:
- Use an Elasticsearch database that have the current valid state
- All write requests are received by a "Microservice Gateway" that:
- Validate the request and the new state that is requested
- Produce a write operation in the message broker
- Receives state change events from the message broker
- Responds to requester with the new state changed
- All write operations are consumed by a "Microservice Processor" that:
- Handles all the business logic and data denormalization
- Updates the state in the Elasticsearch database
- Produce a state change event in the message broker
- All read requests are handled by the "Microservice Gateway" that:
- Searchs at the Elasticsearch database for the requested resource's records
- Responds to the requester with the results
My questions:
- This model does some coupling of the domains? The Gateway is validating subresources ID's to ensure data consistency in one Elasticsearch database (the case of A pointing to B).
- I don't know if this model fits reports requests, there are some complex reports that process a lot of data with input parameters from the user and from his point of view, the operation must be "synchronous" (request/response REST)
- Is the validation of requests/new state a part of the business logic (related to DDD)? If so, my model is incorret to separate them into two microservices?
EDIT
My new proposal for my client:
- Instead of having a gateway acting as a part of the microservice, let gateway only for: routing (microservice registry), balancing and auth stuffs (calling a dedicated microservice for authentication/authorization)
- The microservices hold their own data/database, consistency is ensured by event sourcing architeture
- Reports: if it's about one domain, can be a resource at the microservice that holds the data, if more than one domain is required, another microservice will provide the report