Table storage showing data in only string format

0
votes

I'm using ADF pipeline to copy data from data lake to blob storage and then from blob storage to table storage.

enter image description here

As you can see below, here are the column types in ADF Data Flow Sink - Blob Storage (integer, string, timestamp):

enter image description here

Here is the Mapping settings in Copy data activity:

enter image description here

On checking the output in table storage, I see all columns are of string type:

enter image description here

Why is table storage saving data in string values? How do I resolve this issue in table storage so that it will accept columns in the right type (integer, string, timestamp)? Please let me know. Thank you!

2
azure-data-factoryazure-table-storageazure-data-factory-2

2 Answers

2
votes

In usually, when load data from blob storage in Data Factory, all the default data type in blob file are String, Data Factory will help you convert the data type automatically to Sink.

But it also can not meet all our requests.

I tested copy data from Blob to Table Storage and found that: if we don't specify the data type manually in Source, after pipeline executed, all the data type will be String in Sink(Table Storage).

For example, this my Source blob file: enter image description here

If I don't change the source data type, it seems that everything is ok in Sink table: enter image description here enter image description here

But after the pipeline executed, the data type in table storage are all String: enter image description here

If we change the data type in Source blob manually, and it works ok! enter image description here

enter image description here

For another question, a little confuse that just from you screenshot, that seems the UI of Mapping Data Flow Sink, but Mapping Data Flow doesn't support Table Storage as Sink.

Hope this helps.

0
votes

Finally figured out the issue - I was using DelimitedText format for Blob Storage. After converting to Parquet format, I can see data being written to table storage in correct type.