Terraform unable to use third party providers

Question

Description: I am trying to use an Elasticsearch provider for Terraform. Since there is no official one from Elastic or from Hashicorp I am trying to use a community one "https://registry.terraform.io/providers/phillbaker/elasticsearch/latest".

Terraform version: Terraform v0.14.4

Code:

I tried to put everything in 1 .tf file. I also tried to create a separate module for the resources like Hashicorp recommends. Both methods generate the same error message.

terraform {
  required_providers {
    elk = {
      source  = "phillbaker/elasticsearch"
      version = "1.5.1"
    }
  }
}

provider "elk" {
  url = "https://<my_elk_server>"
}

resource "elasticsearch_index" "index" {
  name = var.elasticsearch_index_name
}

Problem:

terraform init isn't able to find the appropriate provider in the Terraform Registry for some reason.

Initializing the backend...

Initializing provider plugins...
- Finding latest version of hashicorp/elasticsearch...
- Finding phillbaker/elasticsearch versions matching "1.5.1"...
- Installing phillbaker/elasticsearch v1.5.1...
- Installed phillbaker/elasticsearch v1.5.1 (self-signed, key ID 02AD42CD82B6A957)

Partner and community providers are signed by their developers.
If you'd like to know more about provider signing, you can read about it here:

Error: Failed to query available provider packages
https://www.terraform.io/docs/plugins/signing.html

Could not retrieve the list of available versions for provider
hashicorp/elasticsearch: provider registry registry.terraform.io does not have
a provider named registry.terraform.io/hashicorp/elasticsearch

If you have just upgraded directly from Terraform v0.12 to Terraform v0.14
then please upgrade to Terraform v0.13 first and follow the upgrade guide for
that release, which might help you address this problem.

No tfstate files are being generated.

How do I use third party providers from the Terraform Registry ?

Martin Atkins Martin Atkins · Accepted Answer · 2021-01-14T18:41:45

In your required_providers block you've told Terraform that you intend to refer to this provider as "elk" within this module:

    elk = {
      source  = "phillbaker/elasticsearch"
      version = "1.5.1"
    }

Typically you'd set the local name of the provider to be the same as the "type" portion of the provider source address, like this:

    elasticsearch = {
      source  = "phillbaker/elasticsearch"
      version = "1.5.1"
    }

If you change the local name in this way then use references to elasticsearch elsewhere in the module should then refer to the community provider as you intended.

Note that means you'll also need to change the provider block so it has a matching local name:

provider "elasticsearch" {
  url = "https://<my_elk_server>"
}

A different approach here would be to continue to use elk as the name and then change the rest of the configuration to properly refer to that non-default name. I don't recommend doing this, because typically I'd expect the local name to only mismatch the type in the unusual case where your module depends on two providers with the same type name, but I'm mentioning this in the hope that it helps to understand how the Terraform language infers provider dependencies when not given explicitly:

terraform {
  required_providers {
    elk = {
      source  = "phillbaker/elasticsearch"
      version = "1.5.1"
    }
  }
}

# "elk" here is matched with the local names in the
# required_providers block, so this will work.
provider "elk" {
  url = "https://<my_elk_server>"
}

# This "elasticsearch_" prefix causes Terraform to look
# for a provider with the local name "elasticsearch"
# by default...
resource "elasticsearch_index" "index" {
  # ...so if you've given the provider a different local
  # name then you need to associate the resource with
  # the provider configuration explicitly:
  provider = elk

  name = var.elasticsearch_index_name
}

I expect most Terraform users would find the above approach surprising, so in the interests of using familiar Terraform idiom I'd suggest instead following my first suggestion of renaming the local name to elasticsearch, which will then allow the automatic resource-to-provider association to work.

Terraform unable to use third party providers

2 Answers