1
votes

Inserting/Querying to local U-SQL tables is quite fast as compared to when we are creating and using tables from ADLA. We were able to insert data from a CSV to three tables locally in around 8 minutes without using any parallelism and same job took around 10 minutes on ADLA with parallelism of 10 (without setting parallelism job was taking much longer time). Can we achieve same performance?

1

1 Answers

0
votes

What are the table definitions and data sizes? In particular, how did you specify the clustering and fine grade partition distributions?

Also, the compilation time of a local job can be much shorter than in the cluster, because it does not need to create a scale out plan.