Have to delete environment variables for aws cli to work without --profile flag

Question

ok so I am baffled by this aws cli behavior. Basically what is going on is that when I set my AWS creds related in environment variable, AWS CLI forces me to pass --profile flag each time I use the CLI.

So basically when AWS_ACCESS_KEY_ID AND AWS_SECRET_ACCESS_KEY then I cannot run commands like aws s3 ls without passing --profile flag to it even though my profile is [default]

Also, jus to note the environment variable values and the values inside my /.aws/credentials file is exactly same. Also, I tried to set both AWS_PROFILE and AWS_DEFAULT_PROFILE to default hoping that if all values such as keys,secret and profile are set in environment variable then I do not have to pass any --profile flag explicitly. Not having to pass this flag explicitly is very important for me at this point because if I am running an application which connects with aws and picks up default credentials, there is no easy way to pass profile information to that app.

my credentials file look like following:

[default]
aws_access_key_id = AKIA****
aws_secret_access_key = VpR***

My config file looks like following:

[default]
region = us-west-1
output = json

And my environment variables do have the same values for corresponding entries. for key, secret and profile at least.

Any idea on how to solve this issue?

How does you ~/.aws/config look like? How do you environment variables look like? — Dunedan
You should set both AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY, or neither. There's no situation in which setting just one of them is useful. This works fine and you should not have to provide the --profile option. You need to double-check that you have actually set the values correctly in your environment. — jarmod
@Dunedan, I updated my question with the information that you asked for. — Lost
@jarmod I did have both set up. I am removing OR part from my question to alleviate any further confusion. — Lost

sloppypasta sloppypasta · Accepted Answer · 2020-01-14T20:25:53

The AWS CLI looks for credentials using a series of providers in a particular order. (https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html#config-settings-and-precedence)

Specifically:

Command line options – You can specify --region, --output, and --profile as parameters on the command line.

Environment variables – You can store values in the environment variables: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_SESSION_TOKEN. If they are present, they are used.

CLI credentials file – This is one of the files that is updated when you run the command aws configure. The file is located at ~/.aws/credentials on Linux or macOS, or at C:\Users\USERNAME\.aws\credentials on Windows. This file can contain the credential details for the default profile and any named profiles.

CLI configuration file – This is another file that is updated when you run the command aws configure. The file is located at ~/.aws/config on Linux or macOS, or at C:\Users\USERNAME\.aws\config on Windows. This file contains the configuration settings for the default profile and any named profiles.

Container credentials – You can associate an IAM role with each of your Amazon Elastic Container Service (Amazon ECS) task definitions. Temporary credentials for that role are then available to that task's containers. For more information, see IAM Roles for Tasks in the Amazon Elastic Container Service Developer Guide.

Instance profile credentials – You can associate an IAM role with each of your Amazon Elastic Compute Cloud (Amazon EC2) instances. Temporary credentials for that role are then available to code running in the instance. The credentials are delivered through the Amazon EC2 metadata service. For more information, see IAM Roles for Amazon EC2 in the Amazon EC2 User Guide for Linux Instances and Using Instance Profiles in the IAM User Guide.

Another potential option for you would be to unset any colliding variables in your env and rely on the aws credentials file to provide the appropriate access credentials from the default entry.

Have to delete environment variables for aws cli to work without --profile flag

1 Answers