Troubleshooting Issues with the Stream Analytics

Question

We have an Azure Stream Analytics configured with input data configured from an Event Hub and output is configured to be written to Table Storage.

The even hub gets the messages from the API (custom code written within the API to send messages to eventhub.

The issue which we are facing is Stream Analytics is getting stopped once in a while and we don't have any trace of error. We are clueless about the reason for the failure. Definitely the input format might be incorrect. Is there any way we can see the messages that is present in the eventhub?

Answer 1

In case you have the chance to use blob containers within your storage account, there might be two possible solutions to begin with:

1. Configure "Diagnostics logs" for the Stream Analytics component and log everything to a blob container within your storage account. You can specifically activate three different settings: Execution, Authoring and AllMetrics. The corresponding blob containers will be created automatically by Stream Analytics. I was able to find errors within the Execution container in my storage account in the past.
2. You can define an Input (named for example Raw-Storage-Input) retrieving all Event Hub messages and write its data into a Blob Container Output (named 'Raw-Storage-Output') within your storage account by doing something along the lines of SELECT * INTO [Raw-Storage-Output] FROM [Raw-Storage-Input];. By doing this you might be able to even write the faulty messages to your blob container before Stream Analytics fails. However, this might not always work reliably.

There are probably more sophisticated ways I'm not aware of, but these options provided some help in the past for me.