How do I transfer my build output files to an Azure VM using PowerShell DSC?

Question

I've been toying around with DSC and I think it's an awesome platform. I made a few tests to automate the deployment of our TFS build outputs and to automatically install the web applications and configure an environment.

This was relatively easy, as I could pass my drop folder path to the DSC script using a file share on our internal network, and use relative paths inside the configuration to select each of our modules.

My problem now is in how to expand this to Azure virtual machines. We wanted to create these scripts to automatically deploy to our QA and Production servers, which are hosted on Azure. Since they are not in our domain, I can't use the File resource anymore to transfer the files, but at the same time I want exactly the same functionality: I'd like to somehow point the configuration to our build output folder and copy the files from there to the virtual machines.

Is there some way I can copy the drop folder files easily from inside a configuration that is run on these remote computers, without sharing the same network and domain? I successfully configured the VMs to accept DSC calls over https using certificates, and I just found out that the Azure PowerShell cmdlets enable you to upload a configuration to Azure storage and run it in the VMs automatically (which seems a lot better than what I did) but I still don't know how I'd get access to my build outputs from inside the virtual machine when the configuration script is run.

Release Management's DSC implementation puts the build outputs in an Azure storage account. Did you look into doing that? — Daniel Mann
@DanielMann I did not know RM did that, interesting. Is it documented somewhere? This is what I'll probably end up doing, now that I think about it. I just need to add my drop files to the zip that is published to azure storage somehow, and then I wouldn't need direct access to the drops. I would like to stay away from the new pipelines because they only support the deployment tasks... this is not enough to me, I need to run arbitrary actions on the workflow, like I mentioned in another question, for deploying my android apks to google play for example. — julealgon
I don't think it's documented anywhere, but presumably it's using the Azure SDK to push the binaries from the build drop/RM server into Azure storage, then executing something on the deployment target to download the binaries from Azure storage. The Azure APIs can definitely do this, so it's just a matter of hacking your own together. — Daniel Mann

julealgon julealgon · Accepted Answer · 2014-10-30T13:05:10

I ended up using the Publish-AzureVMDscExtension cmdlet to create a local zip file, appending my build outputs to the zip, and then publishing the zip, something along those lines:

function Publish-AzureDscConfiguration
{
    [CmdletBinding()]
    Param(
        [Parameter(Mandatory)]
        [string] $ConfigurationPath
    )

    Begin{}
    Process
    {
        $zippedConfigurationPath = "$ConfigurationPath.zip";

        Publish-AzureVMDscConfiguration -ConfigurationPath:$ConfigurationPath -ConfigurationArchivePath:$zippedConfigurationPath -Force

        $tempFolderName = [System.Guid]::NewGuid().ToString();
        $tempFolderPath = "$env:TEMP\$tempFolderName";
        $dropFolderPath = "$tempFolderPath\BuildDrop";

        try{
            Write-Verbose "Creating temporary folder and symbolic link to build outputs at '$tempFolderPath' ...";
            New-Item -ItemType:Directory -Path:$tempFolderPath;
            New-Symlink -LiteralPath:$dropFolderPath -TargetPath:$PWD;
            Invoke-Expression ".\7za a $tempFolderPath\BuildDrop.zip $dropFolderPath -r -x!'7za.exe' -x!'DscDeployment.ps1'";

            Write-Verbose "Adding component files to DSC package in '$zippedConfigurationPath'...";
            Invoke-Expression ".\7za a $zippedConfigurationPath $dropFolderPath.zip";
        }
        finally{
            Write-Verbose "Removing symbolic link and temporary folder at '$tempFolderPath'...";
            Remove-ReparsePoint -Path:$dropFolderPath;
            Remove-Item -Path:$tempFolderPath -Recurse -Force;
        }
        Publish-AzureVMDscConfiguration -ConfigurationPath:$zippedConfigurationPath -Force
    }
    End{}
}

By using a zip inside the zip used by Azure, I can access the inner contents in the working directory of the PowerShell DSC Extension (in the DSCWork folder). I tried just adding the drop folder directly to the zip (without zipping it first), but then the DSC Extension copies the folder to the modules path, thinking it is a module.

I'm not completely happy with this solution yet, and I'm having a few problems already, but it makes sense in my mind and should work fine.

How do I transfer my build output files to an Azure VM using PowerShell DSC?

1 Answers