AWS-CLI credentials not found using RStudio, but works fine from Windows CLI

Question

When I issue an AWS-CLI command from the windows command line, "aws s3 ls" it successfully completes. When I try shell(paste0("aws ", "s3 ", "ls")) using R with Eclipse, it successfully completes.

However, on another machine using the same credentials, from the windows command line, "aws s3 ls" successfully completes, but with RStudio shell(paste0("aws ", "s3 ", "ls")) I get:

"fatal error: Unable to locate credentials"

If I run shell(paste0("aws ", "s3 ", "ls ", "--debug")) from RStudio I get:

2017-02-27 15:07:10,516 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: env 2017-02-27 15:07:10,516 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: assume-role 2017-02-27 15:07:10,516 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: shared-credentials-file 2017-02-27 15:07:10,516 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: config-file 2017-02-27 15:07:10,516 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: ec2-credentials-file 2017-02-27 15:07:10,516 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: boto-config 2017-02-27 15:07:10,516 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: container-role 2017-02-27 15:07:10,516 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: iam-role 2017-02-27 15:07:10,520 - MainThread - botocore.vendored.requests.packages.urllib3.connectionpool - INFO - Starting new HTTP connection (1): xxx.xxx.xxx.xxx 2017-02-27 15:07:10,523 - MainThread - botocore.utils - DEBUG - Caught exception while trying to retrieve credentials: ('Connection aborted.', error(10051, 'A socket operation was attempted to an unreachable network')) Traceback (most recent call last): File "botocore\utils.pyc", line 159, in _get_request File "botocore\vendored\requests\api.pyc", line 69, in get File "botocore\vendored\requests\api.pyc", line 50, in request File "botocore\vendored\requests\sessions.pyc", line 465, in request File "botocore\vendored\requests\sessions.pyc", line 573, in send File "botocore\vendored\requests\adapters.pyc", line 415, in send

For the successful case from the Windows command line, it finds the credentials as seen below:

2017-02-27 15:07:03,267 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: env 2017-02-27 15:07:03,267 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: assume-role 2017-02-27 15:07:03,267 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: shared-credentials-file 2017-02-27 15:07:03,267 - MainThread - botocore.credentials - INFO - Found credentials in shared credentials file: ~/.aws/credentials

So, what I am trying to figure out is why the AWS CLI cannot find the credentials using RStudio, but it works fine from the Windows CLI.

UPDATE: I have installed Eclipse on both machines and it works fine. However, it is still not working with RStudio, so it must be an IDE-related issue. Any ideas?

You might try instead using the aws.s3 package, which gives you a native R interface to S3. — Thomas
Thank you for the suggestion. Unfortunately, I am no longer working on this project and cannot try it out. Seems like a good option though. — Jeff

Maciej Milewski Maciej Milewski · Accepted Answer · 2018-02-07T16:45:04

I was able to solve it by defining AWS credentials in RStudio:

Sys.setenv ("AWS_ACCESS_KEY_ID" = "YourKEY_ID",
           "AWS_SECRET_ACCESS_KEY" = "YOURSECRETACCESSKEY",
           "AWS_DEFAULT_REGION" = "eu-west-1")

AWS-CLI credentials not found using RStudio, but works fine from Windows CLI

1 Answers