Preventing partition column from appearing in exported data

Question

I have an external table partitioned on Timestamp column which is of datetime type. So the external table definition looks like this:-

.create external table external_mytable (mydata:dynamic,Timestamp:datetime)
kind=blob
partition by bin(Timestamp,1d)
dataformat=json
( 
    h@'https://<mystorage>.blob.core.windows.net/<mycontainer>;<storagekey>'
)

The source table for the export is mytable which has a bunch of columns but I am only interested in a column called mydata holding actual payload and other columns year, month & day, which are required to drive partitioning.

My export looks like this:-

.export async to table  external_mytable <| mysourcetable | project mydata,Timestamp=make_datetime(year,month,day)

Now in this case I don't ideally want Timestamp column to be part of actual exported JSON data. I am forced to specify it because this column is driving partitioning logic. Is there any way to avoid Timestamp appearing in the exported data and still be used in determining partitioning in this case?

Avnera Avnera · Accepted Answer · 2020-02-23T04:51:31

Thanks for the ask Dhiraj, this is on our backlog. Feel feel to open similar asks on our user voice where we can update once it is complete.

Preventing partition column from appearing in exported data

1 Answers