Terraform back-end to azure blob storage errors

13
votes

I have been using the below to successfully create a back-end state file for terraform in Azure storage, but for some reason its stopped working. I've recycled passwords for the storage, trying both keys and get the same error every-time

backend.tf

    terraform {
    backend "azurerm" {
        storage_account_name    = "terraformstorage"
        resource_group_name     = "automation"
        container_name          = "terraform" 
        key                     = "testautomation.terraform.tfstate"
        access_key              = "<storage key>"
    }
}

Error returned

terraform init
Initializing the backend...
Successfully configured the backend "azurerm"! Terraform will automatically use this backend unless the backend configuration changes.
Error refreshing state: storage: service returned error: StatusCode=403, ErrorCode=AuthenticationFailed, ErrorMessage=Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature. RequestId:665e0067-b01e-007a-6084-97da67000000
Time:2018-12-19T10:18:18.7148241Z, RequestInitiated=Wed, 19 Dec 2018 10:18:18 GMT, RequestId=665e0067-b01e-007a-6084-97da67000000, API Version=, QueryParameterName=, QueryParameterValue=

Any ideas what im doing wrong?

5
azureterraform
It looks like your credentials are wrong. Can you try accessing the state file in the blob storage via another mechanism? Does Azure have a useful CLI tool you can use to test this? - ydaetskcoR
hi ydaetskcoR, I can connect successfully to the storage using "Azure Storage Explorer" using storage account name and key, which is pretty much all that terraform uses. just to add to the confusion! - Staggerlee011
Another option to explore: make sure the container_name match the container used to generate the sas_token - Lolorol

5 Answers

15
votes

What worked for me is to delete the local .terraform folder and try again.

11
votes

Another problem can be time resolution.

I experienced those problems as well, tried all the above mentioned steps, but nothing helped.

What happened on my system (Windows 10, WSL2) was, that WSL lost its time sync and I was hours apart. This behaviour is described in https://github.com/microsoft/WSL/issues/4245.

For me it helped to

  • get the appropriate time in WSL (sudo hwclock -s) and

  • to reboot WSL

Hope, this will help others too.

4
votes

Here are few suggestions:

  • Run: terraform init -reconfigure.
  • Confirm your "terraform/backend" credentials.
  • In case your Terraform contains some "azurerm_storage_account/network_rules" to allow certain IP addresses, or make sure you're connected to the right VPN network.
  • If above won't work, run TF_LOG=TRACE terraform init to debug further.
2
votes

Please ensure you've been authenticated properly to Azure Cloud.

If you're running Terraform externally, re-run: az login.

If you're running Terraform on the instance, you can use managed identities, or by defining the following environmental variables:

ARM_USE_MSI=true
ARM_SUBSCRIPTION_ID=xxx-yyy-zzz
ARM_TENANT_ID=xxx-yyy-zzz

or just run az login --identity, then assign the right role (azurerm_role_assignment, e.g. "Contributor") and appropriate policies (azurerm_policy_definition).

See also:

-1
votes

There should a .terraform directory , where you are running the terraform init command from. Remove .terraform or move it to Someotehr name. Next time terraform init runs , it will recreate that directory with new init.