Azure Event Hub log sorting streaming using query

1
votes

How to process Azure Log events from Event hub and filter based on criteria.

We are trying to filter specific critical or security related Diagnostic and Activity logs before feeding into our Onprem SIEM solution.

Can someone please guide me how to filter the data from Event hub and then re-ingest into another event hub. Whether this is possible or any other alternatives available out there.

At a high level , the flow is shown below. Source from Diagnostic Logs (Monitor) -> Event Hub -> Filter/Query -> Event Hubenter image description here

1
azure-logic-appsazure-eventhubazure-log-analyticsazure-diagnosticsazure-security
What type of filter would you like to apply? If payload is JSON or AVRO and want to filter one of the properties of the document then you can write Azure Stream Analytics queries to forward events to another eventhub. – Serkant Karaca
The payload is JSON from Azure event logs, and I have already tried the Stream analytics , which is not able to sort the data as required. – Anil Kash

1 Answers

0
votes

The answer could be very bare - but in our case we are using Azure Functions to perform similar operations. As you know the shape of data - the function can decide if event is forwarded to other EH or it is dropped.