Create blob storage container via Azure data factory

Question

I am working on a project which uses Azure data factory and Blob Storage. I have a requirement but not sure how to implement it.

Requirement:

#1 Check if blob storage container exists. #2 Create of new container in Data Factory via a power shell script. Basicaly, the container will only be created the first time that the pipeline is run. #3 Call API.

#1==> i found way to check if blob exists, but did not find container.

#2==> What is the best way to create a container from Azure Data Factory via a power shell script : via a batch process ? Or is it better to do it via a function?

Thank you very much for your help in advance

Hi @GuillaumeLabs If my answer is helpful for you, please accept(mark) it as answer. This can be beneficial to other community members. Very thanks! — Joseph Xu

Joseph Xu Joseph Xu · Accepted Answer · 2020-12-08T10:00:59

As the post said, we can use GetMetadata activity and If Condition activity to check if blob exists. If blob does not exist we can use Azure function to create the container and specify the name of the container through http get request. I think using Azure function is more convenient.

For example, here I'm using nodejs in Azure function. According to Azure function javascript and Manage blobs with JavaScript v12 SDK in Node.js. We can start a project.

Add dependencies to the nodejs demo.

Then modify some code index.js so we can manage the blob container(Create a container).

const { BlobServiceClient } = require('@azure/storage-blob');


module.exports = async function (context, req) {

    const AZURE_STORAGE_CONNECTION_STRING = "<your_connect_string>";
    const blobServiceClient = BlobServiceClient.fromConnectionString(AZURE_STORAGE_CONNECTION_STRING);

    const name = (req.query.name || (req.body && req.body.name));

    const containerClient = blobServiceClient.getContainerClient(name);

    // Create the container
    const createContainerResponse = await containerClient.create();

    const responseMessage = name
        ? "Container:" + name + " has been created!"
        : "...";

    context.res = {
        // status: 200, /* Defaults to 200 */
        body: responseMessage
    };
}

After deployed to Azure function, I tested the code, it works well. The container was created.

Create blob storage container via Azure data factory

1 Answers