2
votes

I try to use Azure-Data-Explorer to create time-series of IoT data that is stored in Azure-Data-Lake-Gen2. The folder structure in Azure-Data-Lake is "archive/yyyy/MM/dd/HH/*.json".

The idea is:

  1. Create an external table and json mapping (https://docs.microsoft.com/en-us/azure/data-explorer/data-lake-query-data)
  2. Make time-series (https://docs.microsoft.com/en-us/azure/data-explorer/time-series-analysis)
  3. Use analytics to get cool insights ;-)

I've tried for step one (in the Azure-Data-Explorer query):

.create external table ExternalAdlGen2 (timestamp:datetime, timestamp2:datetime, deviceId:string, telemetry:dynamic)
kind=adl
partition format_datetime="'year='yyyy/'month='MM/'day='dd/'hour='HH" by bin(timestamp, 1h)
dataformat=json
( 
   @h'abfss://filesystem@<storage acount name>.dfs.core.windows.net/archive/path;sharedkey=<access key>'
)

and also:

.create external table ExternalAdlGen2 (timestamp:datetime, timestamp2:datetime, deviceId:string, telemetry:dynamic)
kind=adl
partition format_datetime="'year='yyyy/'month='MM/'day='dd/'hour='HH" by bin(timestamp, 1h)
dataformat=json
( 
   @h'abfss://filesystem@<storage acount name>.dfs.core.windows.net/archive/path;sharedkey=<access key>'
)
with 
(
   docstring = "Docs",
   folder = "ExternalTables",
   namePrefix="Prefix"
) 

The result is the following error message: "Error I could not parse that, sorry. clientRequestId: KustoWebV2; some_guid"

Do you have any suggestion how to do that properly?

Thanks in advance.

1

1 Answers

1
votes

there was a bug in the documentation which will be fixed shortly.

in the URL specification - please replace @h'abfss://... with h@'abfss://...