I try to use Azure-Data-Explorer to create time-series of IoT data that is stored in Azure-Data-Lake-Gen2. The folder structure in Azure-Data-Lake is "archive/yyyy/MM/dd/HH/*.json".
The idea is:
- Create an external table and json mapping (https://docs.microsoft.com/en-us/azure/data-explorer/data-lake-query-data)
- Make time-series (https://docs.microsoft.com/en-us/azure/data-explorer/time-series-analysis)
- Use analytics to get cool insights ;-)
I've tried for step one (in the Azure-Data-Explorer query):
.create external table ExternalAdlGen2 (timestamp:datetime, timestamp2:datetime, deviceId:string, telemetry:dynamic)
kind=adl
partition format_datetime="'year='yyyy/'month='MM/'day='dd/'hour='HH" by bin(timestamp, 1h)
dataformat=json
(
@h'abfss://filesystem@<storage acount name>.dfs.core.windows.net/archive/path;sharedkey=<access key>'
)
and also:
.create external table ExternalAdlGen2 (timestamp:datetime, timestamp2:datetime, deviceId:string, telemetry:dynamic)
kind=adl
partition format_datetime="'year='yyyy/'month='MM/'day='dd/'hour='HH" by bin(timestamp, 1h)
dataformat=json
(
@h'abfss://filesystem@<storage acount name>.dfs.core.windows.net/archive/path;sharedkey=<access key>'
)
with
(
docstring = "Docs",
folder = "ExternalTables",
namePrefix="Prefix"
)
The result is the following error message: "Error I could not parse that, sorry. clientRequestId: KustoWebV2; some_guid"
Do you have any suggestion how to do that properly?
Thanks in advance.