Azure Data Factory get data from foreach value

Question

I've a lookup activity with gets value from a container within a blob. I've a foreach activity which is connected to the lookup activity

and under settings it has the value:

@activity('LookupMessageType').output.value

I've another copy activity which runs within this foreach activity. It copies data from cosmos DB to Azure Data Lake.

This is the query in the source dataset:

select c.Tag, data.assetTag, data.timestamp, data.jsonVersion, data.messageType, data.speed from c join data in c.data
where (data.speed>  item().speed_Value) AND
(data.timestamp >= '@{formatDateTime(addhours(pipeline().TriggerTime, -1), 'yyyy-MM-ddTHH:mm:ssZ' )}' 
AND data.timestamp < '@{formatDateTime(pipeline().TriggerTime, 'yyyy-MM-ddTHH:mm:ssZ' )}')

I get an error as I run this pipeline:

{
    "errorCode": "2200",
    "message": "Failure happened on 'Source' side. ErrorCode=UserErrorDocumentDBReadError,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=DocumentDb operation failed: Message: {\"errors\":[{\"severity\":\"Error\",\"location\":{\"start\":231,\"end\":235},\"code\":\"SC2005\",\"message\":\"'item' is not a recognized built-in function name.\"}]}\r\nActivityId: *redacted*, documentdb-dotnet-sdk/1.21.1 Host/64-bit MicrosoftWindowsNT/6.2.9200.0.,Source=Microsoft.DataTransfer.ClientLibrary.DocumentDb,''Type=Microsoft.Azure.Documents.BadRequestException,Message=Message: {\"errors\":[{\"severity\":\"Error\",\"location\":{\"start\":231,\"end\":235},\"code\":\"SC2005\",\"message\":\"'item' is not a recognized built-in function name.\"}]}\r\nActivityId: redacted, documentdb-dotnet-sdk/1.21.1 Host/64-bit MicrosoftWindowsNT/6.2.9200.0,Source=Microsoft.Azure.Documents.Client,''Type=System.Runtime.InteropServices.COMException,Message=Exception from HRESULT: 0x800A0B00,Source=,'",
    "failureType": "UserError",
    "target": "Copy Data1"
}

It is saying item is not a built in function name. I want to use value from the foreach activity to query the source.

Furthermore, here is my dynamic filepath content

@concat('test_', item().speed, '.csv')

I get the desired dynamic file structure with this expression, if I am using static values in the query like: data.speed> 500

MAK MAK · Accepted Answer · 2018-09-13T19:21:51

So, I figured out what was wrong with the query: instead of

where (data.speed> item().speed_Value)

I had to use

where (data.speed> @{item().speed_value})

Azure Data Factory get data from foreach value

1 Answers