Can we add range of partitions to a table in ADL from dynamic data

Question

Is it possible to add partitions dynamically instead of fixed to specific static data. For example, if we need to create partitions for all dates from different CSV records.

Answer 1

You have to create the partition using ALTER TABLE ADD PARTITION explicitly (after creating the partitioned table) today. So the current suggestion is to look at all distinct dates of your data and generate the ALTER statement programmatically.

I suggest you add a request to http://aka.ms/adlfeedback for a more dynamic partition generation.

Answer 2

You can pass dynamic data (dates is the classic example) to create partitions, sample construct below, does this help?

E.g.

CREATE TABLE MyTable(Day DateTime, MyValue string, ....,
                     INDEX idx CLUSTERED(MyValue)
                     PARTITIONED BY BUCKETS(Day)
                     HASH(MyValue) INTO 100
);