0
votes

I am uploading JSONs to Azure Blob storage using the Azure Blob storage API's function:

    const response = await blobClient.upload(content, content.length);

There is absolutely no compression logic in the code nor any encoding headers being added but the files seem to be around 60% of their original size when they reach the storage. Also, monitoring the PUT requests using fiddler it seems that the file is compressed and then uploaded by the API.

My question is, does Azure do compression by default?

EDIT: I was stringifying and then uploading the json objects. They get all the white-spaces remove and hence the reduced size.

2
I believe gzip is used by default. Could identity encoding be used instead? I can give it a shot and do some tests if you want. Let me know.LMG
I don't believe SDK would be compressing the files by default. Are you able to download the files and reopen them? Please edit your question and include the complete code?Gaurav Mantri

2 Answers

2
votes

Based on my test, there is no compression problem. Here is my sample:

const { BlobServiceClient } = require("@azure/storage-blob");
var fs = require('fs');

async function main() {
    const AZURE_STORAGE_CONNECTION_STRING = "Your_Stroage_Account_Connection_String";
    const blobServiceClient = BlobServiceClient.fromConnectionString(AZURE_STORAGE_CONNECTION_STRING);

    const containerName = 'demo';
    const blobName = 'test.txt';

    const containerClient = blobServiceClient.getContainerClient(containerName);
    if(!await containerClient.exists()){
        await containerClient.create();
    }
    const contents = fs.readFileSync('test.txt');
    const blockBlobClient = containerClient.getBlockBlobClient(blobName);
    await blockBlobClient.upload(contents,contents.length);

}

main().then(() => console.log('Done')).catch((ex) => console.log(ex.message));

The test.txt file's size is about 99.9KB.

enter image description here

And, from the portal, the uploaded file's size is 99.96KB,which is in line with our expectations.

enter image description here

0
votes

You should also use byte length when uploading, as storage blob api expects number of bytes, the string length can be different

  const content = "Hello 世界!";
  console.log(`length: ${content.length}`);
  console.log(`byteLength: ${Buffer.byteLength(content)}`);

the output:

length: 9
byteLength: 15