Azure Event Grid Custom Topic Triggering on Insert to Azure SQL table

Question

I am trying to use Event Grid to kick off an Azure Data Factory pipeline when a new record is inserted into an Azure SQL database table. But, I'm lost at the start of things.

When creating the new subscription, I think I would choose Custom Input Schema but I'm not sure where to even start with where to get the "Event Type" from. Is there a list of types somewhere? Is this in the documentation of Azure SQL or Event Grid?

What is the right event type? Any help would be appreciated.

Reference: https://docs.microsoft.com/en-us/azure/event-grid/event-sources

NOTE: I cannot use Logic Apps for this as that has not been approved by our Azure architecture team. I say this because Logic Apps SQL connector now allows for a trigger based on SQL table insert -- no matter though, because I cannot use Logic Apps :(

Answer 1

At this moment, SQL Database doesn't publish events to Event Grid, so you can't use this approach.

You can change your code and right after insert on SQL, publish a custom event to Event Grid, or switch to CosmosDB which offers the Change Feed (which you can subscribe and react to events)

Answer 2

Yes, At the moment there is no event sync integration to the Azure Event grid, however, for purpose of exploring other venues, you might find Debezium a place for syncing most of the data sources into a Kafka or other streams with little custom code.

reference SQL - Sever Debezium Connector

Note: the few of the connectors are in testing phase, and it is a little time taking task to write or customize the connector, but can be done.

I felt this technology mostly usefull for integrating or migrating complex distributed systems with legacy components working together.