I have a pipeline with a Copy activity which copies data from a folder in ADLS Gen2 with Parquet files to a table in Azure SQL Database. Sometimes the Copy activity is in status Queued for several minutes before the actual copy of data occurs and then succeeds. The Copy activity has a pre-copy script setting for the Sink that truncates the table before copying the data. Why is the Copy activity queued for a long time before copying the data? Can it be due to some lock in the database, i.e. that some query is reading from the table?
1
votes
How many pipelines do you have? Did you triggered the pipeline run or manually execute the pipeline?
- Leon Yue
In this data factory resource, I only have one pipeline and I triggered it manually when I got the Status=Queued issue.
- Magnus Johannesson
which integration runtime did you use? please try to change or restart it.
- Leon Yue
DefaultIntegrationRuntime (West Europe). I assume I cannot restart it?
- Magnus Johannesson
Sorry, we only can change and refresh it. How about using a self integration runtime?
- Leon Yue
1 Answers
1
votes
Below is similar issue with Azure IR, after looking at the logs internally, it is addressed as an issue occurred due to ADF related service outage in West Europe. Internal team has confirmed that the issue has been resolved.
Could you please re-run the pipeline and let us know if you still notice the issue.
Related MSDN case: copy activities on Azure IR has long queue wait times
