Use pre-installed Terraform plugins instead of downloading them with terraform init

Question

While running terraform init when using Terraform 0.11.3 we are getting the following error:

Initializing provider plugins... - Checking for available provider plugins on https://releases.hashicorp.com...

Error installing provider "template": Get https://releases.hashicorp.com/terraform-provider-template/: read tcp 172.25.77.25:53742->151.101.13.183:443: read: connection reset by peer.

Terraform analyses the configuration and state and automatically downloads plugins for the providers used. However, when attempting to download this plugin an unexpected error occured.

This may be caused if for some reason Terraform is unable to reach the plugin repository. The repository may be unreachable if access is blocked by a firewall.

If automatic installation is not possible or desirable in your environment, you may alternatively manually install plugins by downloading a suitable distribution package and placing the plugin's executable file in the following directory: terraform.d/plugins/linux_amd64

I realized it's because of connectivity issues with https://releases.hashicorp.com domain. For some obvious reasons, we will have to adjust with this connectivity issue as there are some SSL and firewall issues between the control server and Hashicorp's servers.

Is there any way we could bypass this by downloading the plugins from Hashicorp's servers and copying them onto the control server? Or any other alternative to avoid trying to download things from Hashicorp's servers?

Answer 1

You can use pre-installed plugins by either putting the plugins in the same directory as the terraform binary or by setting the -plugin-dir flag.

It's also possible to build a bundle of every provider you need automatically using the terraform-bundle tool.

I run Terraform in our CI pipeline in a Docker container so have a Dockerfile that looks something like this:

FROM golang:alpine AS terraform-bundler-build

RUN apk --no-cache add git unzip && \
    go get -d -v github.com/hashicorp/terraform && \
    go install ./src/github.com/hashicorp/terraform/tools/terraform-bundle

COPY terraform-bundle.hcl .

RUN terraform-bundle package terraform-bundle.hcl && \
    mkdir -p terraform-bundle && \
    unzip -d terraform-bundle terraform_*.zip

####################

FROM python:alpine

RUN apk add --no-cache git make && \
    pip install awscli

COPY --from=terraform-bundler-build /go/terraform-bundle/* /usr/local/bin/

Note that the finished container image also adds git, make and the AWS CLI as I also require those tools in the CI jobs that uses this container.

The terraform-bundle.hcl then looks something like this (taken from the terraform-bundle README):

terraform {
  # Version of Terraform to include in the bundle. An exact version number
  # is required.
  version = "0.10.0"
}

# Define which provider plugins are to be included
providers {
  # Include the newest "aws" provider version in the 1.0 series.
  aws = ["~> 1.0"]

  # Include both the newest 1.0 and 2.0 versions of the "google" provider.
  # Each item in these lists allows a distinct version to be added. If the
  # two expressions match different versions then _both_ are included in
  # the bundle archive.
  google = ["~> 1.0", "~> 2.0"]

  # Include a custom plugin to the bundle. Will search for the plugin in the 
  # plugins directory, and package it with the bundle archive. Plugin must have
  # a name of the form: terraform-provider-*, and must be build with the operating
  # system and architecture that terraform enterprise is running, e.g. linux and amd64
  customplugin = ["0.1"]
}

Answer 2

config plugin_cache_dir in .terraformrc

plugin_cache_dir   = "$HOME/.terraform.d/plugin-cache"

then move the pre-installed provider into the plugin_cache_dir,

terraform will not download the provider anymore

btw, use the ~/.terraform.d/plugin directory doesn't work

/.terraform.d/plugin/linux_amd64$ terraform -v
Terraform v0.12.15

Answer 3

Starting 0.13.2 version of Terraform release, one could download plugins from a local webserver/http server via network mirror protocol.

For more details, check this link

It expects a .terraformrc file in $HOME path, pointing to the provider path of the plugins like below. If the file is in different directory, you could provide the path with TERRAFORM_CONFIG env var.

provider_installation {
  network_mirror {
    url    = "https://terraform-plugins.example.net/providers/"
  }
}

Then, you define providers in a custom tf like below.

providers.tf::

terraform {
  required_providers {
    azurerm = {
      source  = "registry.terraform.io/example/azurerm"
    }
    openstack = {
      source  = "registry.terraform.io/example/openstack"
    }
    null = {
      source  = "registry.terraform.io/example/null"
    }
    random = {
      source  = "registry.terraform.io/example/random"
    }
    local = {
      source  = "registry.terraform.io/example/local"
    }
  }
}

However, you have to upload the plugin file in .zip format along with index.json and the <version>.json files for terraform to discover the version of plugin to download.

Example index.json containing the version of plugin::

{
  "versions": {
    "2.3.0": {}
  }
}

Again, 2.3.0.json contains hashes of the plugin file. In this case it's <version>.json

{
  "archives": {
    "linux_amd64": {
      "hashes": [
        "h1:nFL6uiwsQFLiP8QCr35sPfWe9LpXI3/c7gP9tYnih+k="
      ],
      "url": "terraform-provider-random_2.3.0_linux_amd64.zip"
    }
  }
}

How do you get details of index.json and <version>.json files?

By running terraform providers on the directory containing tf files. Note, the machine running this command, needs to connect to public terraform registry. Terraform will download the information of these files. If you have different terraform configuration files, it makes sense to automate these steps otherwise, you could manually do :)

Upon, terraform init, terraform downloads the plugins from above web server rather from terraform registry. Make sure you don't use plugin-dir argument with terraform init as it will override all the changes you made.

Answer 4

Updated Dockerfile for @ydaetskcoR 's solution, because currently terraform-bundle doesn't work with 0.12.x (the problem was fixed at 0.12.2, but appeared on 0.12.18)

FROM hashicorp/terraform:0.12.18 as terraform-provider

COPY provider.tf .

RUN terraform init && \
    mv .terraform/plugins/linux_amd64/terraform-provider* /bin/ 

FROM hashicorp/terraform:0.12.18
# Install terraform pre-installed plugins
COPY --from=terraform-provider /bin/terraform-provider* /bin/

And here is the content of provider.tf

provider "template" { version = "~>2.1.2" }
provider "aws" { version = "~>2.15.0" }
...

Answer 5

This took me awhile, had the same problem. I ended up having to download from source and use the image that this spits out. Its nasty, but it does what i need it do to to work with the Google provider.

FROM golang:alpine AS terraform-bundler-build

ENV TERRAFORM_VERSION=0.12.20
ENV GOOGLE_PROVIDER=3.5.0

RUN apk add --update --no-cache git make tree bash curl
ENV GOPATH=/go
RUN mkdir -p $GOPATH/src/github.com/terraform-providers

RUN cd $GOPATH/src/github.com/terraform-providers && curl -sLO https://github.com/terraform-providers/terraform-provider-google-beta/archive/v$GOOGLE_PROVIDER.tar.gz
RUN cd $GOPATH/src/github.com/terraform-providers && tar xvzf v$GOOGLE_PROVIDER.tar.gz && mv terraform-provider-google-beta-$GOOGLE_PROVIDER terraform-provider-google-beta
RUN cd $GOPATH/src/github.com/terraform-providers/terraform-provider-google-beta && pwd &&  make build

RUN cd $GOPATH/src/github.com/terraform-providers && curl -sLO https://github.com/terraform-providers/terraform-provider-google/archive/v$GOOGLE_PROVIDER.tar.gz
RUN cd $GOPATH/src/github.com/terraform-providers && tar xvzf v$GOOGLE_PROVIDER.tar.gz && mv terraform-provider-google-$GOOGLE_PROVIDER terraform-provider-google
RUN cd $GOPATH/src/github.com/terraform-providers/terraform-provider-google && pwd &&  make build

RUN mkdir -p $GOPATH/src/github.com/hashicorp
RUN cd $GOPATH/src/github.com/hashicorp && curl -sLO https://github.com/hashicorp/terraform/archive/v$TERRAFORM_VERSION.tar.gz
RUN cd $GOPATH/src/github.com/hashicorp && tar xvzf v$TERRAFORM_VERSION.tar.gz && mv terraform-$TERRAFORM_VERSION terraform
RUN cd $GOPATH/src/github.com/hashicorp/terraform && go install ./tools/terraform-bundle
ENV TF_DEV=false
ENV TF_RELEASE=true
COPY my-build.sh $GOPATH/src/github.com/hashicorp/terraform/scripts/
RUN cd $GOPATH/src/github.com/hashicorp/terraform && /bin/bash scripts/my-build.sh
ENV HOME=/root
COPY terraformrc $HOME/.terraformrc
RUN mkdir -p $HOME/.terraform.d/plugin-cache

########################################
FROM alpine:3
ENV HOME=/root

RUN ["/bin/sh", "-c", "apk add --update --no-cache bash ca-certificates curl git jq openssh"]

RUN ["bin/sh", "-c", "mkdir -p /src"]
COPY --from=terraform-bundler-build /go/bin/terraform* /bin/
RUN mkdir -p /root/.terraform.d/plugins/linux_amd64
COPY --from=terraform-bundler-build /root/.terraform.d/ $HOME/.terraform.d/
RUN cp /bin/terraform-provider-google $HOME/.terraform.d/plugin-cache/linux_amd64
RUN cp /bin/terraform-provider-google-beta $HOME/.terraform.d/plugin-cache/linux_amd64

COPY terraformrc $HOME/.terraformrc

COPY provider.tf $HOME/
COPY backend.tf $HOME/

# For Testing (This should be echoed or taken care of in the CI pipeline)
#COPY google.json $HOME/.google.json
WORKDIR $HOME
ENTRYPOINT ["/bin/bash"]

.terraformrc:

plugin_cache_dir   = "$HOME/.terraform.d/plugins/linux_amd64"
disable_checkpoint = true

provider.tf

# Define which provider plugins are to be included
provider "google" {
  credentials = ".google.json"
}

provider "google-beta" {
  credentials = ".google.json"
}

my-build.sh

#!/usr/bin/env bash
#
# This script builds the application from source for multiple platforms.

# Get the parent directory of where this script is.
SOURCE="${BASH_SOURCE[0]}"
while [ -h "$SOURCE" ] ; do SOURCE="$(readlink "$SOURCE")"; done
DIR="$( cd -P "$( dirname "$SOURCE" )/.." && pwd )"

# Change into that directory
cd "$DIR"
echo "DIR=$DIR"

# Get the git commit
GIT_COMMIT=$(git rev-parse HEAD)
GIT_DIRTY=$(test -n "`git status --porcelain`" && echo "+CHANGES" || true)

# Determine the arch/os combos we're building for
XC_ARCH=${XC_ARCH:-"amd64 arm"}
XC_OS=${XC_OS:-linux}
XC_EXCLUDE_OSARCH="!darwin/arm !darwin/386"

mkdir -p bin/

# If its dev mode, only build for ourself
if [[ -n "${TF_DEV}" ]]; then
    XC_OS=$(go env GOOS)
    XC_ARCH=$(go env GOARCH)

    # Allow LD_FLAGS to be appended during development compilations
    LD_FLAGS="-X main.GitCommit=${GIT_COMMIT}${GIT_DIRTY} $LD_FLAGS"
fi

if ! which gox > /dev/null; then
    echo "==> Installing gox..."
    go get -u github.com/mitchellh/gox
fi

# Instruct gox to build statically linked binaries
export CGO_ENABLED=0

# In release mode we don't want debug information in the binary
if [[ -n "${TF_RELEASE}" ]]; then
    LD_FLAGS="-s -w"
fi

# Ensure all remote modules are downloaded and cached before build so that
# the concurrent builds launched by gox won't race to redundantly download them.
go mod download

# Build!
echo "==> Building..."
gox \
    -os="${XC_OS}" \
    -arch="${XC_ARCH}" \
    -osarch="${XC_EXCLUDE_OSARCH}" \
    -ldflags "${LD_FLAGS}" \
    -output "pkg/{{.OS}}_{{.Arch}}/${PWD##*/}" \
    .


## Move all the compiled things to the $GOPATH/bin
GOPATH=${GOPATH:-$(go env GOPATH)}
case $(uname) in
    CYGWIN*)
        GOPATH="$(cygpath $GOPATH)"
        ;;
esac
OLDIFS=$IFS
IFS=: MAIN_GOPATH=($GOPATH)
IFS=$OLDIFS
#
# Create GOPATH/bin if it's doesn't exists
if [ ! -d $MAIN_GOPATH/bin ]; then
    echo "==> Creating GOPATH/bin directory..."
    mkdir -p $MAIN_GOPATH/bin
fi

# Copy our OS/Arch to the bin/ directory
DEV_PLATFORM="./pkg/$(go env GOOS)_$(go env GOARCH)"
if [[ -d "${DEV_PLATFORM}" ]]; then
    for F in $(find ${DEV_PLATFORM} -mindepth 1 -maxdepth 1 -type f); do
        cp ${F} bin/
        cp ${F} ${MAIN_GOPATH}/bin/
        ls -alrt ${MAIN_GOPATH}/bin/
        echo "MAIN_GOPATH=${MAIN_GOPATH}"
    done
fi

bucket.tf

terraform {
  backend "gcs" {
    bucket = "my-terraform-bucket"
    prefix = "terraform/state"
    credentials = ".google.json"
  }
  required_version = "v0.12.20"
}

Answer 6

You can use pre-installed plugins by either putting the plugins binaries in the same directory where Terraform binary is available by setting the "plugins-dir" flag.

By default, all plugins downloaded in .terraform folder. For example, Null resource plugin will be available at below location

.terraform\providers\registry.terraform.io\hashicorp\null\3.0.0.\windows_amd64.

Create new folder like "terraform-plugins" inside Terraform directory and copy all content including registry.terraform.io folder mentioned in above example in created folder.

Now run the terraform init command with plugins-dir flag

terraform init -plugin-dir="/terraform-plugins"

specify complete directory path with plugin-dir flag