Using materialized view on a kusto table

0
votes

I have a scenario where a table gets ingested with 20k+ messages every 3 minutes from different sources. I want to fetch latest data which is sent by each unique source, source can be a an id. ill be able to write a query like this.

FactResources | summarize arg_max(Timestamp, *) by SubscriptionId, ResourceId
  • I want to understand doing this will impact CPU performance of the cluster, considering my cluster is running at 70% CPU utilization now?
  • What will be latency for materialized to view to get refreshed after every ingestion?
  • From java SDK can I call this materialized view just like calling a kusto function?
1
azure-data-explorerkql

1 Answers

5
votes
  • Yes, the CPU of the cluster will be impacted, as materialization consumes CPU (just like all other operations running on the cluster). You can estimate the amount of resources consumed by the materialization process using .show commands-and-queries command, as documented here.
  • There is no guarantee on the latency of materialization. Materialization will run as long as there are records in the delta (what is delta?) and as long as the cluster has available capacity to run materialization. The latency, referred to as materialized view age, depends on the ingestion pattern, the data volume and the available resources in the cluster. You can check the materialized view age by running .show materialized-view and checking the MaterializedTo value. See more in performance considerations and monitoring.
  • You can query the view just like you query a table (ViewName) or by using the materialized-view() function. See materialized views queries.