Azure Stream Analytics GetMetadataPropertyValue not working

Question

I have a .net application simulating an iot device. It is using Microsoft.Azure.Devices.Client to create device messages and send them to the iothub which then routes them to an event hub where Azure Stream Analytics has it as an input.

I have the following in the "simulator":

  messageToSend = new Message(Encoding.ASCII.GetBytes(JsonConvert.SerializeObject(message.Payload)))
                 {
                   MessageId = new Guid().ToString(),
                   ContentType = "application/json",
                   ContentEncoding = "utf-8",
                 };
                 messageToSend.Properties.Add("test", "test");

In my azure stream analytics query i have:

SELECT *, GetMetadataPropertyValue(InputEH, '[User].[test]') as test
INTO OutputEH 
FROM InputEH
WHERE test = 'test' 

For some reason the query never outputs any data to OutputEH. I have tried the same query except instead of adding the "test" property to the MessageProperties i have put it in the message body and it works as expected if my query uses "WHERE InputEh.test = 'test'"

To validate the properties I set the output to an eventhub and set up an event hub processor on it and as the messages come in if I remove the "WHERE" clause. In the event hub processor it appears that the headers have been removed, this tells me ASA is dropping the headers for some reason.

Not sure what I'm doing wrong here. Any help would be appreciated.

Answer 1

It turns out the problem was in the query. GetMetaDataPropertyValue() works but not as it's described on MSDN @ GetMetaDataPropertyValue. I had to move the method into my where clause in order to get it to work successfully. So now my query is the following:

SELECT *
INTO OutputEH 
FROM InputEH as input
WHERE GetMetadataPropertyValue(input, '[User].[test]') = test

I'm not sure why doing it in the SELECT clause isn't working but this worked for me.

Answer 2

For anyone else that runs into this "problem" you can also get what you want by doing something like this instead:

WITH AllData AS (
    SELECT
        utctime, 
        GetMetadataPropertyValue(EventHubInput, '[User].[StreamID]') as streamid, 
        GetMetadataPropertyValue(EventHubInput, '[User].[CustomerID]') as customerid, 
        message
    FROM
        EventHubInput timestamp by utcTime
)

SELECT *
INTO EventOutput
FROM AllData
WHERE customerId = '<some value>' and streamId = '<some other value>'

Answer 3

In your last post, GetMetadataPropertyValue(input, '[User].[test]') is null, and test also is null. So in the query string, where section is always true, and this means that it will output all the data from InputEH to OutputEH. And more,you need to test when you are running the job using live data from EventHub.