How to save byte[] as string in U-SQL

Question

SQL script (Azure Data Lake Analytics) in which I extract too large text for string. So I use byte[]. But when I save results in CSV file, this text is BASE64 encoded.

Is there a option to save it as simple string? (For saving I use Outputters.Csv() ).

OR:

Then I copy data (with Azure Data Factory) into SQL database. Is there option to store this text as varchar/text datatype in database? I found that DF has decodeBase64() function https://docs.microsoft.com/en-us/azure/data-factory/control-flow-expression-language-functions#conversion-functions but I do not know how to use it while copying.

Answer 1

Actually the built-in CSV outputter uses hex-encoding and not base64 encoding. Although that could be a useful option. The Csv/Tsv/Text extractor/outputter family expects that the data is text based and that non character code points like ETX or LF are only used sparingly for delimiting purposes. Thus instead of writing an arbitrary byte stream, it will serialize it hex encoded.

If you want to write it out without hex encoding, I suggest that you write a custom outputter that will write the byte[] as is. Or use a hex-decoder on the consuming side. I probably would try the latter before writing the outputter (although it is probably a simple enough outputter).