1
votes

I'm trying to create a simple way for product owners to stand up and tear down the Azure resources required for the Virtual Assistant. However, the deployment script deploy.ps1 requires PowerShell 6+, which doesn't seem to be available with Azure DevOps.

What's the best way to proceed?

Describe the solution you'd like

The goal is to use a release pipeline to automatically stand up the Azure resources required for the bot to function.

Thoughts

  • Is there a way to use PowerShell 6+ using Azure PowerShell?
  • Should we not use deploy.ps1 to deploy the resources in a DevOps release pipeline?

Code

Here's the contents of deploy.ps1 and a link to the latest.

#Requires -Version 6

Param(
    [string] $name,
    [string] $resourceGroup,
    [string] $location,
    [string] $appId,
    [string] $appPassword,
    [string] $luisAuthoringKey,
    [string] $luisAuthoringRegion,
    [string] $parametersFile,
    [string] $languages = "en-us",
    [string] $outFolder = $(Get-Location),
    [string] $logFile = $(Join-Path $PSScriptRoot .. "deploy_log.txt")
)

# Reset log file
if (Test-Path $logFile) {
    Clear-Content $logFile -Force | Out-Null
}
else {
    New-Item -Path $logFile | Out-Null
}

# Get mandatory parameters
if (-not $name) {
    $name = Read-Host "? Bot Name (used as default name for resource group and deployed resources)"
}

if (-not $resourceGroup) {
    $resourceGroup = $name
}

if (-not $location) {
    $location = Read-Host "? Azure resource group region"
}

if (-not $appPassword) {
    $appPassword = Read-Host "? Password for MSA app registration (must be at least 16 characters long, contain at least 1 special character, and contain at least 1 numeric character)"
}

if (-not $luisAuthoringRegion) {
    $luisAuthoringRegion = Read-Host "? LUIS Authoring Region (westus, westeurope, or australiaeast)"
}

if (-not $luisAuthoringKey) {
    Switch ($luisAuthoringRegion) {
        "westus" { 
            $luisAuthoringKey = Read-Host "? LUIS Authoring Key (found at https://luis.ai/user/settings)"
            Break
        }
        "westeurope" {
            $luisAuthoringKey = Read-Host "? LUIS Authoring Key (found at https://eu.luis.ai/user/settings)"
            Break
        }
        "australiaeast" {
            $luisAuthoringKey = Read-Host "? LUIS Authoring Key (found at https://au.luis.ai/user/settings)"
            Break
        }
        default {
            Write-Host "! $($luisAuthoringRegion) is not a valid LUIS authoring region." -ForegroundColor DarkRed
            Break
        }
    }

    if (-not $luisAuthoringKey) {
        Break
    }
}

if (-not $appId) {
    # Create app registration
    $app = (az ad app create `
        --display-name $name `
        --password $appPassword `
        --available-to-other-tenants `
        --reply-urls 'https://token.botframework.com/.auth/web/redirect')

    # Retrieve AppId
    if ($app) {
        $appId = ($app | ConvertFrom-Json) | Select-Object -ExpandProperty appId
    }

    if(-not $appId) {
        Write-Host "! Could not provision Microsoft App Registration automatically. Review the log for more information." -ForegroundColor DarkRed
        Write-Host "! Log: $($logFile)" -ForegroundColor DarkRed
        Write-Host "+ Provision an app manually in the Azure Portal, then try again providing the -appId and -appPassword arguments. See https://aka.ms/vamanualappcreation for more information." -ForegroundColor Magenta
        Break
    }
}

# Get timestamp
$timestamp = Get-Date -f MMddyyyyHHmmss

# Create resource group
Write-Host "> Creating resource group ..."
(az group create --name $name --location $location) 2>> $logFile | Out-Null

# Deploy Azure services (deploys LUIS, QnA Maker, Content Moderator, CosmosDB)
Write-Host "> Deploying Azure services (this could take a while)..." -ForegroundColor Yellow
if ($parametersFile) {
    (az group deployment create `
        --name $timestamp `
        --resource-group $resourceGroup `
        --template-file "$(Join-Path $PSScriptRoot '..' 'Resources' 'template.json')" `
        --parameters "@$($parametersFile)" `
        --parameters microsoftAppId=$appId microsoftAppPassword="`"$($appPassword)`"") 2>> $logFile | Out-Null
}
else {
    (az group deployment create `
        --name $timestamp `
        --resource-group $resourceGroup `
        --template-file "$(Join-Path $PSScriptRoot '..' 'Resources' 'template.json')" `
        --parameters microsoftAppId=$appId microsoftAppPassword="`"$($appPassword)`"") 2>> $logFile | Out-Null
}

# Get deployment outputs
$outputs = (az group deployment show `
    --name $timestamp `
    --resource-group $resourceGroup `
    --query properties.outputs)

# If it succeeded then we perform the remainder of the steps
if ($outputs)
{
    # Log and convert to JSON
    $outputs >> $logFile
    $outputs = $outputs | ConvertFrom-Json

    # Update appsettings.json
    Write-Host "> Updating appsettings.json ..."
    if (Test-Path $(Join-Path $outFolder appsettings.json)) {
        $settings = Get-Content $(Join-Path $outFolder appsettings.json) | ConvertFrom-Json
    }
    else {
        $settings = New-Object PSObject
    }

    $settings | Add-Member -Type NoteProperty -Force -Name 'microsoftAppId' -Value $appId
    $settings | Add-Member -Type NoteProperty -Force -Name 'microsoftAppPassword' -Value $appPassword
    if ($outputs.appInsights) { $settings | Add-Member -Type NoteProperty -Force -Name 'appInsights' -Value $outputs.appInsights.value }
    if ($outputs.storage) { $settings | Add-Member -Type NoteProperty -Force -Name 'blobStorage' -Value $outputs.storage.value }
    if ($outputs.cosmosDb) { $settings | Add-Member -Type NoteProperty -Force -Name 'cosmosDb' -Value $outputs.cosmosDb.value }
    if ($outputs.contentModerator) { $settings | Add-Member -Type NoteProperty -Force -Name 'contentModerator' -Value $outputs.contentModerator.value }

    $settings | ConvertTo-Json -depth 100 | Out-File $(Join-Path $outFolder appsettings.json)

    # Delay to let QnA Maker finish setting up
    Start-Sleep -s 30

    # Deploy cognitive models
    Invoke-Expression "$(Join-Path $PSScriptRoot 'deploy_cognitive_models.ps1') -name $($name) -luisAuthoringRegion $($luisAuthoringRegion) -luisAuthoringKey $($luisAuthoringKey) -qnaSubscriptionKey $($outputs.qnaMaker.value.key) -outFolder $($outFolder) -languages `"$($languages)`""

    Write-Host "> Done."
}
else
{
    # Check for failed deployments
    $operations = az group deployment operation list -g $resourceGroup -n $timestamp | ConvertFrom-Json
    $failedOperations = $operations | Where { $_.properties.statusmessage.error -ne $null }
    if ($failedOperations) {
        foreach ($operation in $failedOperations) {
            switch ($operation.properties.statusmessage.error.code) {
                "MissingRegistrationForLocation" {
                    Write-Host "! Deployment failed for resource of type $($operation.properties.targetResource.resourceType). This resource is not avaliable in the location provided." -ForegroundColor DarkRed
                    Write-Host "+ Update the .\Deployment\Resources\parameters.template.json file with a valid region for this resource and provide the file path in the -parametersFile parameter." -ForegroundColor Magenta
                }
                default {
                    Write-Host "! Deployment failed for resource of type $($operation.properties.targetResource.resourceType)."
                    Write-Host "! Code: $($operation.properties.statusMessage.error.code)."
                    Write-Host "! Message: $($operation.properties.statusMessage.error.message)."
                }
            }
        }

        Write-Host "+ To delete this resource group, run 'az group delete -g $($resourceGroup) --no-wait'" -ForegroundColor Magenta
        Break
    }
}

Screenshots of Setup

enter image description here

enter image description here

Attempting to Run Azure PowerShell

When I attempt to run the script using Azure PowerShell, I receive the following error.

##[error]The script 'deploy.ps1' cannot be run because it contained a "#requires" statement for Windows PowerShell 6.0. The version of Windows PowerShell that is required by the script does not match the currently running version of Windows PowerShell 5.1.17763.316.

This is with Azure PowerShell Version set to Latest installed version. Using Specify other version and 6.7.0, 6.2.1 or 6.0.0 also doesn't work. None of these seem to actually have an effect. It always come back as 5.1.17763.316.

2
Don't you have a ARM template file in JSON format added with the project template that can create the resources? also what is the content of the deploy.ps1?Kasun Kodagoda
I edited my post to include the contents of deploy.ps1. I might have an ARM template somewhere that's usable, but I think does more than what I'd get out of an ARM template. I'm not sure. How to proceed isn't mentioned in the Virtual Assistant docs. Any tips are appreciated.Eric Hansen
@EricHansen why do you think PowerShell 6+ not available on Azure DevOps?Shayki Abramczyk
Hello @PravinAmbekar. In my professional opinion ~ 1 year after my original post, don't use deploy.ps1 from a pipeline. deploy.ps1 is a great quick start script to get you up and running, but if you want to really make this robust for enterprise use, modify the Microsoft provided ARM templates to suit your needs. I recommend reading through each line of deploy.ps1 in detail. You'll see that it uses a template stored here github.com/microsoft/botframework-solutions/tree/master/….Eric Hansen
I think the best solution is to use an ARM deployment task to deploy/update the resources. You will still need to run some CLI commands to interact with the resources that you provisioned. I recommend NOT using one of the PowerShell scripts provided by Microsoft. Instead, pull out the specific CLI commands that you need---use the scripts as an example to see what you need---and create individual tasks for the individual CLI commands that you need for your specific scenario.Eric Hansen

2 Answers

3
votes

First of all, your script is using Read-Host, which is not usable in the pipeline. Build agents do have pwsh (so Powershell 6.0 +), but your script isn't using Azure Powershell at all, its using Azure CLI (which is also available on the build agent).

In general you'd need to refactor your script for the pipeline and it would work just fine.

3
votes

Build Agents do have PSv6, they also have PSv5 Software on Each Agent is listed at: https://github.com/Microsoft/azure-pipelines-image-generation/tree/master/images/win

The issue here is the task is using "Azure Powershell" which uses the PS v5 cmdlets, thus will force the use of PSv5 (powershell.exe) instead of PSv6 (pwsh.exe).

If you were to use the normal "Powershell" task and select "Use Powershell Core" you would have a hard time (or insecure time) getting logged in.

For this scenario, you need to execute a PSv6 script from the Azure CLI task. To do this, simply add the Azure CLI Task and execute the inline script below (fix path and parameters to what you posted).

pwsh -File ${System.DefaultWorkingDirectory}\<some-path-to>\deploy.ps1 -script-arguments-copy-pasted-here

NOTE: The only way this works is if ALL parameters are passed and read-host isn't hit, as 4c74356b41 stated, you can't get inputs from the pipeline.