How to make a continuous delivery of a python function app deployed in Azure?

Question

For the first time I deployed a Python function app to Azure using a deployment pipeline:

https://docs.microsoft.com/bs-latn-ba/azure/azure-functions/functions-how-to-azure-devops

The package is deployed to Azure using Kudu Zip deploy.

My http triggered function runs wonderfully locally (on Windows), but I have a 500 internal errors on Azure because it does not find the module requests.

Exception: ModuleNotFoundError: No module named 'requests'

imports of __init__.py:

import logging, requests, os
import azure.functions as func

If I remove the 'requests' dependency the function works on Azure (status 200).

The requests library is imported by the requirement.txt and copied to the .venv36/lib/site-packages/requests by the build pipeline.

So I am wondering if the virtual environment .venv36 that is built in the package is used by the function deployed in Azure. There is no indication about how to activate virtual environments in Azure.

Answer 1

If you name your virtual env worker_venv as named in the documentation you linked, it should work (assuming you are using a Linux environment for your pipeline).

However, the Python Azure Functions documentation is to be updated very soon, and the recommended way would be to not deploy the entire virtual environment from your deployment pipeline. Instead, you'd want to install your packages in .python_packages/lib/site-packages.

You could do --

pip3.6 install --target .python_packages/lib/site-packages -r requirements.txt

Instead of --

python3.6 -m venv worker_venv
source worker_venv/bin/activate
pip3.6 install setuptools
pip3.6 install -r requirements.txt

And it should work fine.

Answer 2

We are also having the same issue using the newest version of the YAML pipeline template:

- task: UsePythonVersion@0
  displayName: 'Use Python 3.6'
  inputs:
    versionSpec: 3.6 # Functions V2 supports Python 3.6 as of today

- bash: |
    python -m venv worker_venv
    source worker_venv/bin/activate
    pip install -r requirements.txt
  workingDirectory: $(workingDirectory)
  displayName: 'Install application dependencies'

Removing the virtual environment step, the Function App deployed and run without any issues. This does not seem to be Python best practices; however, it was the only thing we could do to get this deployed correctly on Azure DevOps Pipelines.

Separately, before making this change, we were able to deploy using the Visual Studio code plugin, which indicated to us that this was an environment issue.

Updated docs from Microsoft (1/12/2020)

https://docs.microsoft.com/en-us/azure/azure-functions/functions-how-to-azure-devops?tabs=python

azure-pipelines.yml (our working version on Azure DevOps Pipelines)

- master

variables:
  # Azure Resource Manager connection created during pipeline creation
  azureSubscription: '<subscription-id>'

  # Function app name
  functionAppName: '<built-function-app-name>'

  # Agent VM image name
  vmImageName: 'ubuntu-latest'

  # Working Directory
  workingDirectory: '$(System.DefaultWorkingDirectory)/__app__'

stages:
- stage: Build
  displayName: Build stage

  jobs:
  - job: Build
    displayName: Build
    pool:
      vmImage: $(vmImageName)

    steps:
    - bash: |
        if [ -f extensions.csproj ]
        then
            dotnet build extensions.csproj --runtime ubuntu.16.04-x64 --output ./bin
        fi
      workingDirectory: $(workingDirectory)
      displayName: 'Build extensions'

    - task: UsePythonVersion@0
      displayName: 'Use Python 3.7'
      inputs:
        versionSpec: 3.7 # Functions V2 supports Python 3.6 as of today

    - bash: |
        pip install --upgrade pip
        pip install --target="./.python_packages/lib/site-packages" -r ./requirements.txt
      workingDirectory: $(workingDirectory)
      displayName: 'Install application dependencies'

    - task: ArchiveFiles@2
      displayName: 'Archive files'
      inputs:
        rootFolderOrFile: '$(workingDirectory)'
        includeRootFolder: false
        archiveType: zip
        archiveFile: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
        replaceExistingArchive: true

    - publish: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
      artifact: drop

- stage: Deploy
  displayName: Deploy stage
  dependsOn: Build
  condition: succeeded()

  jobs:
  - deployment: Deploy
    displayName: Deploy
    environment: 'production'
    pool:
      vmImage: $(vmImageName)

    strategy:
      runOnce:
        deploy:

          steps:
          - task: AzureFunctionApp@1
            displayName: 'Azure functions app deploy'
            inputs:
              azureSubscription: '$(azureSubscription)'
              appType: functionAppLinux
              appName: $(functionAppName)
              package: '$(Pipeline.Workspace)/drop/$(Build.BuildId).zip'

Answer 3

It definitely needs to be more clearly pointed out that the proper directory for Python packages when deploying Azure Functions is .python_packages/lib/site-packages. I had to go digging through the Azure Function Core Tools source code to see where they put Python packages.

Also had to dig around in the Function debug console to see where Oryx grabs packages from.

I guess there is a pointer in the Version 3.7 YAML file here, but no callout of the directory's importance and does it apply to Python 3.8 Functions?

If I'm not mistaken, this is a requirement to use DevOps to deploy Python Functions (unless you want to install Function Core Tools as part of your build pipeline!).

Answer 4

You need to handle those 2 imports separately,

import azure.functions as func
import requests

Answer 5

Hopefully I am understanding your problem correctly.

When you are installing on your local machine, libs are installed where python is (or at least somewhere other than where your actual code is). This means, when you package your code, you aren't actually keeping the libs together.

To get around this, you can use a virtual env. Python provide a venv tool (there is also a a standard linux virtual env tool) which you can run via:

python -m venv /path/to/my/dir
source /path/to/my/dir/bin/activate 
cd /path/to/my/dir/bin/activate
pip install -r requirements.txt
deactivate

I know you mentioned windows, so I would suggest using WSL and the ubuntu image (generally a nice tool to have anyway). There probably is a way to get that working in windows otherwise though I don't know it.

EDIT: Fixed format

Answer 6

Although its old but:

*pip(python version) install --target .python_packages/lib/site-packages -r requirements.txt

For ex. if you are using 3.7 then pip3.7 install --target .python_packages/lib/site-packages -r requirements.txt

Works like a charm