Edit the files with replacing string in DevOps using shell script

0
votes

I have created a build pipeline for Azure data factory in DevOps which will fetch the files from an azure repo.

Before deploying the changes i would like to replace few string values in some of the files located in the build artifact.

Below mentioned are the steps i followed.

  1. Added a shell script file in the root folder of repository.
  2. Created a build pipeline pointing to the repository branch which needs to be deployed.
  3. Created a release pipeline which takes the build artifact.
  4. Added an shell script in the agent job for replacing few string values from the file in the repo.

Below is the screenshot for repository structure. enter image description here

After publishing the artifact in the build pipeline, below is the structure of published artifact in the pipeline.

enter image description here

I have added a sample shell script to the Agent job which is already in the artifact, as shown below

enter image description here

while running the release I am getting the below error regarding the path as shown below enter image description here

How to update the files in the artifact by replacing the string values, using power shell. Does anyone have an idea how to point to the files in the artifact and run a replace script. If you could share a sample replace script by selecting the file name, it will be helpful.

3
azureazure-devopsazure-data-factoryazure-data-factory-2
Hi, Antony. I would like provide your another task to achieve your request, you could check my answer for some more details. If you still want to use powershell, please let me know for free. – Leo Liu-MSFT
This was helpful for string replacement during the deployment. – Antony
I have a query that at the time of ARM deployment in the prod/UAT instance, suppose if a pipeline is already running in that instance, what will happen to that pipeline/task? Will it get interrupted by the deployment or is it taken care by DevOps/ADF? @Leo Liu-MSFT – Antony
Would like open new thread for this new question, judging from the current information, I cannot understand your question. You need to open a new post to provide more specific information. I am very willing to help you. – Leo Liu-MSFT
@LeoLiu-MSFT I have created a new thread for the doubt. Please have a look [stackoverflow.com/questions/64531318/… – Antony

3 Answers

1
votes

How to update the files in the artifact by replacing the string values, using power shell. Does anyone have an idea how to point to the files in the artifact and run a replace script.

There is a great extension Replace Tokens to achieve your request.

With this task, you could directly select the file/files you want to replace the strings:

enter image description here

We just need to replace the variables in .json file with format #{VarValue1}# in the repo:

like:

{
    "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "storageAccounts_leoteststorageaccount_name": {
            "defaultValue": "#{TestValue}#",
            "type": "String"
        }
    },

Then define the key's values on the Variables tab:

enter image description here

The test result is:

enter image description here

This method is very simple and does not need to consider the file path, you can try it.

0
votes

Inside the shell script I am not able to go to the repo file location to replace the string value. How to redirect to the root folder of the repo from the shell script file?

Please use workingDirectory to navigate to where you want to have your script executed.

- task: PowerShell@2
  inputs:
    filePath: '$(System.DefaultWorkingDirectory)\scripts\script.ps1'
    workingDirectory: '$(System.DefaultWorkingDirectory)\templates\'

What you need o remember is go give fully qualified location for you script and set working directory for a directory where your arm files are.

If you are looking for other methods you can always consider creating paramaters in your arm templates and instead replacing values just pass them as those paramaetrs.

Is the above mentioned steps are correct for replacing a string value or do we need to publish the build first and then replace file from the artifact?

It is all up to you. You can replace strings in arm files before creating artifact or on release pipeline before running them. The questions you may ask yourself:

  • will I store some secrets in I replace string before publishing artifact
  • when I replace strings will I be able to deploy those string to any env or I need to make replacement once again

However, please consider adding paramaters. Maybe this is the best way.

0
votes

The above error you see in the powershell task in your release pipeline is because you pointed the script path to a .sh1 file, while powershell task is expecting a .ps1 file. Even though the path value you specified in the script path field is correct, powershell task will still fail with error if the file is not a .ps1 file.

If the script is bash script, you should use Bash task instead of powershell task.

You can check out below simple example in powershell script to replace a json value in artifacts file. See this thread.

$json = Convertfrom-json (get-content "_Source_alias\artifactsName\arm_template.json")

$json.parameter.value = $(PipelineVariables)

ConvertTo-Json $json -Depth 4 | Out-File _Source_alias\artifactsName\arm_template.json -Force

If you deploy arm template using ARM template deployment task. This task allows you to replace the template parameters by configuring the Template parameters field and Override template parameters field. See below:

enter image description here