Only use continuous dataset and cut off data, when there is a gap in the data in Kusto

Question

I have a Kusto Query to analyze my app data.

My query shall only show the data from the live app from store, and not the data from our test installations of QA.

So far my only way to distinguish the data is, that the live data is one continuous data every day without gap from a certain date up to now.

The testing data is also generation data, but that is visible as chunks of data for about one or two days in a row and then having a gap.

Here is a screenshot to show how it looks like.

so basically what i want is to cut off all the data before the app went live.

And no, i don't want to manually edit my script each time we go live and change the release date. I want to somehow find out the release date with a sophisticated Kusto query.

Like: Get me all timestamps where every consecutive day has data

i just have no idea how to put this into Kusto Can you guys help me here?

Best Regards, Maverick

Seems like a very complicated way to make a distinction between test data and live data. You could seperate the data by using different log analytic accounts, one for prod and one for test. Or populate a custom property to distinguish the environments. — Peter Bons
i just found out, that there are methods to check weather the app was installed from playstore respectively appstore. i will use this in my log data to distinguish the data. Nevertheless i find the question interesting if this would be possible in kusto the way i described it. so if anyone has a solution for this. feel free :) — Maverick1st

RoyO RoyO · Accepted Answer · 2020-10-11T08:51:08

You can find the last gap in the data using a combination of summarize and prev function, then filter to include only the data after the gap (assuming T is the source dataset):

let lastGap = toscalar(T
  | summarize by Timestamp=bin(Timestamp, 1d)
  | order by Timestamp asc
  | extend gap = Timestamp - prev(Timestamp)
  | where gap > 1d
  | summarize lastGap = max(Timestamp));
T
| where Timestamp >= lastGap

Only use continuous dataset and cut off data, when there is a gap in the data in Kusto

1 Answers