Recovering from an EventStoreException where sequence is already in use within DeadlineHandler

Question

Sometime an EventStoreException occurs mentioning the event couldn't be stored because it has the same sequence as another event in the aggregate.

This happens when EventA and EventB almost have the same timestamp.
CommandA is sent by a controller and CommandB is sent by a Saga within a DeadlineHandler.

So the handling of the deadline fails with the EventStoreException is logged, but not retried.
Would it help if we configure the Saga with a PropagatingErrorHandler?

Events table:

timestamp                      | aggregate_id                         | seq | type
                               |                                      |     |
2020-11-30T15:14:51.345541552Z | b02a5364-ee34-431a-ab1a-6c59bb937845 | 0   | MyAggregate
2020-11-30T15:14:52.06794746Z  | b02a5364-ee34-431a-ab1a-6c59bb937845 | 1   | MyAggregate

Exception details:

org.axonframework.eventsourcing.eventstore.EventStoreException: An event for aggregate [b02a5364-ee34-431a-ab1a-6c59bb937845] at sequence [1] was already inserted

java.sql.BatchUpdateException: Batch entry 0 INSERT INTO events 
(event_id, aggregate_id, sequence_number, type, timestamp, payload_type, payload_revision, payload, metadata) 
VALUES 
('d5be369e-5fd0-475e-b5b6-e12449a4ed04',
'b02a5364-ee34-431a-ab1a -6c59bb937845',
1,
'MyAggregate',
'2020-11-30T15:14:52.067871723Z',
'MyEvent',
NULL,
'{"payload":"payload"}',
'{"metaData":"metaData"}') 
was aborted: ERROR: duplicate key value violates unique constraint "uk_aggregate_identifier_sequence_number"
  Detail: Key (aggregate_id, sequence_number)=(b02a5364-ee34-431a-ab1a-6c59bb937845, 1) already exists.

As you can see the timestamp of the events are nearly the same:
EventA: 2020-11-30T15:14:52.06794746Z vs. EventB: 2020-11-30T15:14:52.067871723Z

Lucas Campos Lucas Campos · Accepted Answer · 2020-12-17T13:17:04

To first answer your question, confiruging a PropagatingErrorHandler does not help because TrackingEventProcessor is not going to retry a DeadlineMessage. It only works for retrying real events which is not the case dor a DeadlineMessage since it is not an Event.

Now to your problem, we are assuming that your Saga has a DeadlineHandler and this component is dispatching a Command towards your Aggregate at the same time another component is also dispatching a Command to the same Aggregate. In that way, the Aggregate is failing to handle the second Command. Based on that, we can give you 2 advices:

Have a consistent Routing Strategy which is used by a Distributed implementation of the CommandBus. In short, it will give you the following:

Two commands with the same routing key will always be routed to the same segment.

Have a RetryScheduler configured on your CommandGateway. You can read more about it here.

The RetryScheduler is capable of scheduling retries when command execution has failed.

Recovering from an EventStoreException where sequence is already in use within DeadlineHandler

1 Answers