We have the following implementation for CloseAsync(PartitionContext, CloseReason) method inside our IEventProcessor:
public async Task CloseAsync(PartitionContext context, CloseReason reason)
{
// log reason and partition id
if (reason == CloseReason.Shutdown)
{
// log shutdown event
await context.CheckpointAsync();
}
}
We know that LeaseLost exceptions are fine in this method and we saw these threads:
Azure EventHub & Function LeaseLostException
What is causing Azure Event Hubs ReceiverDisconnectedException/LeaseLostException?
It used to fail in some calls when the processor was shutting down but it succeeds on others.
Recently, we noticed that it is consistently failing on all calls to CloseAsync() when the reason is CloseReason.Shutdown and always on the exact same exception LeaseLostException. Here is the full exception message and stack trace:
Message: 'The lease ID specified did not match the lease ID for the blob.'
StackTrace:
at Microsoft.Azure.EventHubs.Processor.AzureStorageCheckpointLeaseManager.UpdateLeaseCoreAsync(AzureBlobLease lease) at Microsoft.Azure.EventHubs.Processor.AzureStorageCheckpointLeaseManager.UpdateCheckpointAsync(Lease lease, Checkpoint checkpoint) at Microsoft.Azure.EventHubs.Processor.PartitionContext.PersistCheckpointAsync(Checkpoint checkpoint) at CloseAsync(PartitionContext context, CloseReason reason) at Microsoft.Azure.EventHubs.Processor.PartitionPump.CloseAsync(CloseReason reason)
We are using Microsoft.Azure.EventHubs v4.3.0 and Microsoft.Azure.EventHubs.Processor v4.3.0
We are concerned that in such a case, no checkpoint is written and on the restart of the processor there will be lots of duplicates due to not being able to save a recent checkpoint location. Is there a way to understand what is causing the failures and how to fix them?
