Axon - Duplicate segment claim/unclaimed segments for multiple nodes and multiple databases

Question

We are running a setup locally where we start two instances of an Axon application. The following properties are set in application.yml:

axon:
  eventhandling:
    processors:
      SomeProcessorName:
        initialSegmentCount: 2
        threadCount: 1
        mode: TRACKING

So both nodes have a single thread and they should each process a segment. They both connect to AxonServer. How do the two instances coordinate segment claims?

If I start both of these applications using an in-memory database, I can see in AxonServer that they both attempt to claim segment 0 and that segment 1 is claimed by neither. (We get a duplicated claim/unclaimed segment warning). If they connect to the same database, this does not happen, instance 1 claims segment 0, instance 2 claims segment 1.

Am I then correct in assuming that identical processors have to share a database in order for this to work properly? I can't find this information immediatly in the reference docs.

Does this then also mean that if I would hypothetically want to replicate a projection model for performance reasons (e.g: database server in the US and another one in the EU), this would not work properly?

To clarify: I would want both databases to build an identical query model that could both be queried separately. As it is right now (assuming that we could run two nodes on two databases), node 1 would only process events for segment 0, node 2 would only process events for segment 1. If I understand this correctly, this means that both databases only contain half of the information of the query model.

So in order to pull this off, I would have to create another near-identical codebase, with the only difference being the processor name?

Steven Steven · Accepted Answer · 2019-04-12T09:35:29

I think I can give some guidance in this area.

Axon Server does not provide coordination between Tracking Tokens of TrackingEventProcessor at this point in time. Thus, coordination of this part is purely in your application environment, or differently put, with the Axon Server client.

The most pragmatic approach would be to share the underlying storage solution for your TokenStore between both application; so your assumption on this part is correct. Current implementations of the TokenStore are indeed database-based - nothing stops you to come up with a distributed solution of this though, as this is all open source and freely adjustable.

I do not completely follow your hypothetical suggestion that:

Does this then also mean that if I would hypothetically want to replicate a projection model for performance reasons (e.g: database server in the US and another one in the EU), this would not work properly?

Well, this can work properly, but I think the segmentation of a given TrackingEventProcessor it's TrackingToken is not the way to go in this part. This solution is intended to share the work load of updating a single Query Model. The 'work load' in this scenario is the Event Stream by the way.

If you're looking to replicate a given Query Model by means of reading the Event Stream, I'd indeed suggest to have a second TrackingEventProcessor, which has an identical Event Handling Component underneath. Note that this should not require you to 'replicate the code base'. You should merely need to register two Event Handling Components to two distinct TrackingEventProcessors.

If you are using Spring Boot as configuration, all this is typically abstracted away from you. But if you take a look at the EventProcessingConfigurer, you should be able to find a fair API describing how to achieve this. If things aren't clear in that area, I'd suggest a different issue should be introduced, as the topic somewhat diverges from the original question.

Hoping this is sufficient for you to proceed @MatthiasVanEeghem!

Axon - Duplicate segment claim/unclaimed segments for multiple nodes and multiple databases

1 Answers