Azure backed terraform Error building account

Question

I got suddenly and unexpectedly following error when executing terraform plan.

Error: Error building account: Error getting authenticated object ID: Error parsing json result from the Azure CLI: Error retrieving running Azure CLI: Unable to encode the output with ANSI_X3.4-1968 encoding. U
nsupported characters are discarded.

  on main.tf line 4, in provider "azurerm":
   4: provider "azurerm" {

Log nearby error looks like this:

2020-04-14T10:22:53.257Z [DEBUG] plugin.terraform-provider-azurerm_v2.5.0_x5: Testing if Service Principal / Client Certificate is applicable for Authentication..
2020-04-14T10:22:53.257Z [DEBUG] plugin.terraform-provider-azurerm_v2.5.0_x5: Testing if Multi Tenant Service Principal / Client Secret is applicable for Authentication..
2020-04-14T10:22:53.257Z [DEBUG] plugin.terraform-provider-azurerm_v2.5.0_x5: Testing if Service Principal / Client Secret is applicable for Authentication..
2020-04-14T10:22:53.257Z [DEBUG] plugin.terraform-provider-azurerm_v2.5.0_x5: Testing if Managed Service Identity is applicable for Authentication..
2020-04-14T10:22:53.257Z [DEBUG] plugin.terraform-provider-azurerm_v2.5.0_x5: Testing if Obtaining a token from the Azure CLI is applicable for Authentication..
2020-04-14T10:22:53.257Z [DEBUG] plugin.terraform-provider-azurerm_v2.5.0_x5: Using Obtaining a token from the Azure CLI for Authentication
2020-04-14T10:22:53.258Z [DEBUG] plugin.terraform-provider-azurerm_v2.5.0_x5: [DEBUG] Resource "https://management.core.windows.net/" isn't for the correct Tenant
2020/04/14 10:22:54 [ERROR] <root>: eval: *terraform.EvalConfigProvider, err: Error building account: Error getting authenticated object ID: Error parsing json result from the Azure CLI: Error retrieving running
 Azure CLI: Unable to encode the output with ANSI_X3.4-1968 encoding. Unsupported characters are discarded.
2020/04/14 10:22:54 [ERROR] <root>: eval: *terraform.EvalSequence, err: Error building account: Error getting authenticated object ID: Error parsing json result from the Azure CLI: Error retrieving running Azure
 CLI: Unable to encode the output with ANSI_X3.4-1968 encoding. Unsupported characters are discarded.
2020/04/14 10:22:54 [ERROR] <root>: eval: *terraform.EvalOpFilter, err: Error building account: Error getting authenticated object ID: Error parsing json result from the Azure CLI: Error retrieving running Azure
 CLI: Unable to encode the output with ANSI_X3.4-1968 encoding. Unsupported characters are discarded.
2020/04/14 10:22:54 [ERROR] <root>: eval: *terraform.EvalSequence, err: Error building account: Error getting authenticated object ID: Error parsing json result from the Azure CLI: Error retrieving running Azure
 CLI: Unable to encode the output with ANSI_X3.4-1968 encoding. Unsupported characters are discarded.
2020/04/14 10:22:54 [TRACE] [walkRefresh] Exiting eval tree: provider.azurerm
2020/04/14 10:22:54 [TRACE] vertex "provider.azurerm": visit complete
2020/04/14 10:22:54 [TRACE] dag/walk: upstream of "azurerm_cosmosdb_mongo_database.cupi" errored, so skipping
2020/04/14 10:22:54 [TRACE] dag/walk: upstream of "azurerm_log_analytics_workspace.law-cupi" errored, so skipping
2020/04/14 10:22:54 [TRACE] dag/walk: upstream of "azurerm_cosmosdb_account.cosmodb_account" errored, so skipping
2020/04/14 10:22:54 [TRACE] dag/walk: upstream of "azurerm_cosmosdb_mongo_collection.customer" errored, so skipping
2020/04/14 10:22:54 [TRACE] dag/walk: upstream of "azurerm_resource_group.rg-cupi" errored, so skipping
2020/04/14 10:22:54 [TRACE] dag/walk: upstream of "azurerm_log_analytics_solution.las-cupi" errored, so skipping
2020/04/14 10:22:54 [TRACE] dag/walk: upstream of "azurerm_kubernetes_cluster.aks-cupi" errored, so skipping
2020/04/14 10:22:54 [TRACE] dag/walk: upstream of "azurerm_cosmosdb_mongo_collection.deactivationRequest" errored, so skipping
2020/04/14 10:22:54 [TRACE] dag/walk: upstream of "azurerm_cosmosdb_mongo_collection.customerHash" errored, so skipping
2020/04/14 10:22:54 [TRACE] dag/walk: upstream of "azurerm_cosmosdb_mongo_collection.apiAuth" errored, so skipping
2020/04/14 10:22:54 [TRACE] dag/walk: upstream of "provider.azurerm (close)" errored, so skipping
2020/04/14 10:22:54 [TRACE] dag/walk: upstream of "root" errored, so skipping

and versions of my terraform

$ terraform version
2020/04/14 10:24:24 [INFO] Terraform version: 0.12.24
2020/04/14 10:24:24 [INFO] Go runtime version: go1.12.13
2020/04/14 10:24:24 [INFO] CLI args: []string{"/usr/bin/terraform", "version"}
2020/04/14 10:24:24 [DEBUG] Attempting to open CLI config file: /root/.terraformrc
2020/04/14 10:24:24 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2020/04/14 10:24:24 [INFO] CLI command args: []string{"version"}
Terraform v0.12.24
2020/04/14 10:24:24 [DEBUG] checking for provider in "."
2020/04/14 10:24:24 [DEBUG] checking for provider in "/usr/bin"
2020/04/14 10:24:24 [DEBUG] checking for provider in ".terraform/plugins/linux_amd64"
2020/04/14 10:24:24 [DEBUG] found provider "terraform-provider-azuread_v0.8.0_x4"
2020/04/14 10:24:24 [DEBUG] found provider "terraform-provider-azurerm_v2.5.0_x5"
2020/04/14 10:24:24 [DEBUG] found provider "terraform-provider-random_v2.2.1_x4"
2020/04/14 10:24:24 [DEBUG] found valid plugin: "azurerm", "2.5.0", "/cupi/operations/terraform/.terraform/plugins/linux_amd64/terraform-provider-azurerm_v2.5.0_x5"
2020/04/14 10:24:24 [DEBUG] found valid plugin: "random", "2.2.1", "/cupi/operations/terraform/.terraform/plugins/linux_amd64/terraform-provider-random_v2.2.1_x4"
2020/04/14 10:24:24 [DEBUG] found valid plugin: "azuread", "0.8.0", "/cupi/operations/terraform/.terraform/plugins/linux_amd64/terraform-provider-azuread_v0.8.0_x4"
+ provider.azuread v0.8.0
+ provider.azurerm v2.5.0
+ provider.random v2.2.1

and finally my az cli

$ az --version
azure-cli                          2.3.1

command-modules-nspkg              2.0.3
core                               2.3.1
nspkg                              3.0.4
telemetry                          1.0.4

Python location '/opt/az/bin/python3'
Extensions directory '/root/.azure/cliextensions'

Python (Linux) 3.6.5 (default, Apr  1 2020, 07:19:45)
[GCC 7.5.0]

Legal docs and information: aka.ms/AzureCliLegal

My main.tf file:

provider "azuread" {
  version = "~>0.8"
}
provider "azurerm" {
  version         = "~>2"
  subscription_id = "..."
  features {}
}
terraform {
  backend "azurerm" {}
}

I have also read threads bellow. None of which helped or resolved my issue. Same config that doesn't work today, worked with no modification couple of days ago (only thing that could change on client side are versions of plugins - i tried up/down grades but with no success).

• https://github.com/terraform-providers/terraform-provider-azurerm/issues/3686
• https://github.com/terraform-providers/terraform-provider-azurerm/issues/4906
• Terraform with azure CLI - error building account

Answer 1

As mentioned in the comments, The issue was not providing the service principal in the provider.the correct syntax is :

# Configure the Azure Provider
# https://www.terraform.io/docs/providers/azurerm/index.html
provider "azurerm" {
  subscription_id = var.SUBSCRIPTION_ID
  client_id       = var.SP_CLIENT_ID
  client_secret   = var.SP_CLIENT_SECRET
  tenant_id       = var.SP_TENANT_ID
  version         = "=2.0.0" #Can be overide as you wish
  features {}
}

What is Service principal?

An Azure service principal is an identity created for use with applications, hosted services, and automated tools to access Azure resources. This access is restricted by the roles assigned to the service principal, giving you control over which resources can be accessed and at which level. For security reasons, it's always recommended to use service principals with automated tools rather than allowing them to log in with a user identity.

More info here.

With that been said, Why we should use service principal with Terraform?

1. When using Service principal you can give limited permissions to specific resources.
2. Service Principal is not attached to any user. Therefore, multiple users can use this Service principal.
3. You can assign permissions to the app identity that is different than your own permissions.

Azure Provider: Authenticating using a Service Principal with a Client Secret.

About AZ CLI login issue:

to be honest I don't have an answer that I feel confident to share. But, my guess is that there is an issue with AZ CLI version 2.3.1.

As you can see about 2 weeks ago when the new version released Azure team fixed an issue related to az login so I guess this is why things are acting differently now.

In case you want to check that, you can downgrade to 2.3.0 and check if this is still happening.

Answer 2

As noted in the official documentation for Terraform on how to authenticate using the Azure CLI, it is recommended to authenticate using personal credentials (through the az cli) when running locally.

We recommend using either a Service Principal or Managed Service Identity when running Terraform non-interactively (such as when running Terraform in a CI server) - and authenticating using the Azure CLI when running Terraform locally.

This becomes a little problematic when you would like to run locally in a Docker container, especially since it seems like the output generated by the az cli have changed its output (intentionally or not), so that Terraform can no longer use it.

As Amit already noted in the accepted answer, this seems to be due to a change, but I would argue that it occurred earlier, since I have to roll all the way back to 2.2.0 (2.2.0-1~bionic on ubuntu) to have it working again.

Answer 3

I had the same issue running terraform in a docker container through an ssh client. I managed to fix it with:

export LC_ALL=en_US.UTF-8