Debugging ingestion failure in Kusto

Question

I see a bunch of 'permanent' failures when I fire the following command:-

.show ingestion failures | where FailureKind == "Permanent"

For all the entries that are returned the error code is UpdatePolicy_UnknownError.

The Details column for all the entries shows something like this:-

Failed to invoke update policy. Target Table = 'mytable', Query = '<some query here>': The remote server returned an error: (409) Conflict.: : : 

What does this error mean? How do I find out the root cause behind these failures? The information I find through this command is not sufficient. I also copied OperationId for a sample entry and looked it up against the operations info:-

.show operations | where OperationId == '<sample operation id>'

But all I found in the Status is the message Failed performing non-transactional update policy. I know it failed, but can we find out the underlying reason?

Answer 1

"(409) Conflict" error usually comes from writing to the Azure storage. In general, this error should be treated as a transient one. If it happens in the writing of the main part of the ingestion, it should be retried (****). In your case, it happens in writing the data of the non-transactional update policy - this write is not retried - the data enters the main table, but not the dependent table. In the case of a transactional update policy, the whole ingestion will be failed and then retried.

(****) There was a bug in treating such an error, it was treated as permanent for a short period for the main ingestion data. The bug should be fixed now.