Couldn't access Amazon SQS neither via AWS CLI nor AWS SDK

2
votes

I installed aws cli and configured it. The credentials file is located under ~/.aws/credentials. Contents of it is as follow:

[default]
aws_access_key_id = AKIA........PRYIA
aws_secret_access_key = OsLzxDH........HlPDt39lnY

Also here is my config file under /.aws/config :

[default]
output = json
region = us-west-2

I created a queue under https://sqs.us-west-2.amazonaws.com/. I could send message to the queue in SQS Management Console.

In IAM Management Console i also added AmazonSQSFullAccess policy to my user.

But when i want to list the queues via amazon sqs library(using java) i got the following error:

Access to the resource https://sqs.us-west-2.amazonaws.com/ is denied. (Service: AmazonSQS; Status Code: 403; Error Code: AccessDenied; Request ID: 44c3b1f1-cd8b-5306-b4d3-e0af4bf4ccbe)

So i think there may be something wrong with the library and i tried to access via aws cli with the command "aws sqs list-queues". And it threw the error below :

An error occurred (AccessDenied) when calling the ListQueues operation: Access to the resource https://us-west-2.queue.amazonaws.com/ is denied.

Then i added --debug option and tried to get the queue list again. Here is the output:

aws sqs list-queues --debug
2017-04-05 16:33:06,781 - MainThread - awscli.clidriver - DEBUG - CLI version: aws-cli/1.11.66 Python/2.7.10 Darwin/15.6.0 botocore/1.5.29
2017-04-05 16:33:06,782 - MainThread - awscli.clidriver - DEBUG - Arguments entered to CLI: ['sqs', 'list-queues', '--debug']
2017-04-05 16:33:06,782 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function add_scalar_parsers at 0x10280fa28>
2017-04-05 16:33:06,782 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function inject_assume_role_provider_cache at 0x102530f50>
2017-04-05 16:33:06,783 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/local/Cellar/awscli/1.11.66/libexec/lib/python2.7/site-packages/botocore/data/sqs/2012-11-05/service-2.json
2017-04-05 16:33:06,788 - MainThread - botocore.hooks - DEBUG - Event service-data-loaded.sqs: calling handler <function register_retries_for_service at 0x10210ac08>
2017-04-05 16:33:06,788 - MainThread - botocore.handlers - DEBUG - Registering retry handlers for service: sqs
2017-04-05 16:33:06,788 - MainThread - botocore.hooks - DEBUG - Event building-command-table.sqs: calling handler <function add_waiters at 0x102819b18>
2017-04-05 16:33:06,792 - MainThread - awscli.clidriver - DEBUG - OrderedDict([(u'queue-name-prefix', <awscli.arguments.CLIArgument object at 0x102b97090>)])
2017-04-05 16:33:06,792 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.sqs.list-queues: calling handler <function add_streaming_output_arg at 0x10280fc80>
2017-04-05 16:33:06,792 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.sqs.list-queues: calling handler <function add_cli_input_json at 0x10253ad70>
2017-04-05 16:33:06,792 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.sqs.list-queues: calling handler <function unify_paging_params at 0x102796398>
2017-04-05 16:33:06,796 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/local/Cellar/awscli/1.11.66/libexec/lib/python2.7/site-packages/botocore/data/sqs/2012-11-05/paginators-1.json
2017-04-05 16:33:06,796 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.sqs.list-queues: calling handler <function add_generate_skeleton at 0x102788758>
2017-04-05 16:33:06,796 - MainThread - botocore.hooks - DEBUG - Event before-building-argument-table-parser.sqs.list-queues: calling handler <bound method CliInputJSONArgument.override_required_args of <awscli.customizations.cliinputjson.CliInputJSONArgument object at 0x102b971d0>>
2017-04-05 16:33:06,796 - MainThread - botocore.hooks - DEBUG - Event before-building-argument-table-parser.sqs.list-queues: calling handler <bound method GenerateCliSkeletonArgument.override_required_args of <awscli.customizations.generatecliskeleton.GenerateCliSkeletonArgument object at 0x102af0d90>>
2017-04-05 16:33:06,797 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.sqs.list-queues.queue-name-prefix: calling handler <function uri_param at 0x1023ca320>
2017-04-05 16:33:06,797 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.sqs.list-queues.cli-input-json: calling handler <function uri_param at 0x1023ca320>
2017-04-05 16:33:06,797 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.sqs.list-queues.generate-cli-skeleton: calling handler <function uri_param at 0x1023ca320>
2017-04-05 16:33:06,797 - MainThread - botocore.hooks - DEBUG - Event calling-command.sqs.list-queues: calling handler <bound method GenerateCliSkeletonArgument.generate_json_skeleton of <awscli.customizations.generatecliskeleton.GenerateCliSkeletonArgument object at 0x102af0d90>>
2017-04-05 16:33:06,797 - MainThread - botocore.hooks - DEBUG - Event calling-command.sqs.list-queues: calling handler <bound method CliInputJSONArgument.add_to_call_parameters of <awscli.customizations.cliinputjson.CliInputJSONArgument object at 0x102b971d0>>
2017-04-05 16:33:06,797 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: env
2017-04-05 16:33:06,797 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: assume-role
2017-04-05 16:33:06,797 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: shared-credentials-file
2017-04-05 16:33:06,798 - MainThread - botocore.credentials - INFO - Found credentials in shared credentials file: ~/.aws/credentials
2017-04-05 16:33:06,798 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/local/Cellar/awscli/1.11.66/libexec/lib/python2.7/site-packages/botocore/data/endpoints.json
2017-04-05 16:33:06,811 - MainThread - botocore.client - DEBUG - Registering retry handlers for service: sqs
2017-04-05 16:33:06,812 - MainThread - botocore.hooks - DEBUG - Event creating-client-class.sqs: calling handler <function add_generate_presigned_url at 0x1020d2c80>
2017-04-05 16:33:06,813 - MainThread - botocore.args - DEBUG - The s3 config key is not a dictionary type, ignoring its value of: None
2017-04-05 16:33:06,816 - MainThread - botocore.endpoint - DEBUG - Setting sqs timeout as (60, 60)
2017-04-05 16:33:06,816 - MainThread - botocore.hooks - DEBUG - Event before-parameter-build.sqs.ListQueues: calling handler <function generate_idempotent_uuid at 0x10210a668>
2017-04-05 16:33:06,816 - MainThread - botocore.endpoint - DEBUG - Making request for OperationModel(name=ListQueues) (verify_ssl=True) with params: {'body': {'Action': u'ListQueues', 'Version': u'2012-11-05'}, 'url': u'https://us-west-2.queue.amazonaws.com/', 'headers': {'User-Agent': 'aws-cli/1.11.66 Python/2.7.10 Darwin/15.6.0 botocore/1.5.29'}, 'context': {'client_region': 'us-west-2', 'has_streaming_input': False, 'client_config': <botocore.config.Config object at 0x102cfe310>}, 'query_string': '', 'url_path': '/', 'method': u'POST'}
2017-04-05 16:33:06,817 - MainThread - botocore.hooks - DEBUG - Event request-created.sqs.ListQueues: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x102cfe2d0>>
2017-04-05 16:33:06,817 - MainThread - botocore.auth - DEBUG - Calculating signature using v4 auth.
2017-04-05 16:33:06,817 - MainThread - botocore.auth - DEBUG - CanonicalRequest:
POST
/

host:us-west-2.queue.amazonaws.com
x-amz-date:20170405T133306Z

host;x-amz-date
48a38266faf90970d6c7fea9b15e6ba366e5f6397c2970fc893f8a7b5e207bd0
2017-04-05 16:33:06,817 - MainThread - botocore.auth - DEBUG - StringToSign:
AWS4-HMAC-SHA256
20170405T133306Z
20170405/us-west-2/sqs/aws4_request
1f88edf15f91a626575611ec61a7d614afbc73e004d619fff636cecdb06134b7
2017-04-05 16:33:06,817 - MainThread - botocore.auth - DEBUG - Signature:
52b3d7ed984353964a6528741b8dfd8acef536272541e912fcc5c89f14210252
2017-04-05 16:33:06,820 - MainThread - botocore.endpoint - DEBUG - Sending http request: <PreparedRequest [POST]>
2017-04-05 16:33:06,821 - MainThread - botocore.vendored.requests.packages.urllib3.connectionpool - INFO - Starting new HTTPS connection (1): us-west-2.queue.amazonaws.com
2017-04-05 16:33:08,335 - MainThread - botocore.vendored.requests.packages.urllib3.connectionpool - DEBUG - "POST / HTTP/1.1" 403 321
2017-04-05 16:33:08,338 - MainThread - botocore.parsers - DEBUG - Response headers: {'x-amzn-requestid': 'e107be92-7e65-535e-a29e-efcc8a36f565', 'content-length': '321', 'server': 'Server', 'connection': 'close', 'date': 'Wed, 05 Apr 2017 13:33:07 GMT', 'content-type': 'text/xml'}
2017-04-05 16:33:08,339 - MainThread - botocore.parsers - DEBUG - Response body:
<?xml version="1.0"?><ErrorResponse xmlns="http://queue.amazonaws.com/doc/2012-11-05/"><Error><Type>Sender</Type><Code>AccessDenied</Code><Message>Access to the resource https://us-west-2.queue.amazonaws.com/ is denied.</Message><Detail/></Error><RequestId>e107be92-7e65-535e-a29e-efcc8a36f565</RequestId></ErrorResponse>
2017-04-05 16:33:08,341 - MainThread - botocore.hooks - DEBUG - Event needs-retry.sqs.ListQueues: calling handler <botocore.retryhandler.RetryHandler object at 0x102af0c10>
2017-04-05 16:33:08,342 - MainThread - botocore.retryhandler - DEBUG - No retry needed.
2017-04-05 16:33:08,343 - MainThread - awscli.clidriver - DEBUG - Exception caught in main()
Traceback (most recent call last):
  File "/usr/local/Cellar/awscli/1.11.66/libexec/lib/python2.7/site-packages/awscli/clidriver.py", line 197, in main
    return command_table[parsed_args.command](remaining, parsed_args)
  File "/usr/local/Cellar/awscli/1.11.66/libexec/lib/python2.7/site-packages/awscli/clidriver.py", line 333, in __call__
    return command_table[parsed_args.operation](remaining, parsed_globals)
  File "/usr/local/Cellar/awscli/1.11.66/libexec/lib/python2.7/site-packages/awscli/clidriver.py", line 503, in __call__
    call_parameters, parsed_globals)
  File "/usr/local/Cellar/awscli/1.11.66/libexec/lib/python2.7/site-packages/awscli/clidriver.py", line 622, in invoke
    client, operation_name, parameters, parsed_globals)
  File "/usr/local/Cellar/awscli/1.11.66/libexec/lib/python2.7/site-packages/awscli/clidriver.py", line 634, in _make_client_call
    **parameters)
  File "/usr/local/Cellar/awscli/1.11.66/libexec/lib/python2.7/site-packages/botocore/client.py", line 253, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/Cellar/awscli/1.11.66/libexec/lib/python2.7/site-packages/botocore/client.py", line 543, in _make_api_call
    raise error_class(parsed_response, operation_name)
ClientError: An error occurred (AccessDenied) when calling the ListQueues operation: Access to the resource https://us-west-2.queue.amazonaws.com/ is denied.
2017-04-05 16:33:08,343 - MainThread - awscli.clidriver - DEBUG - Exiting with rc 255

An error occurred (AccessDenied) when calling the ListQueues operation: Access to the resource https://us-west-2.queue.amazonaws.com/ is denied.

As you could see from the debug log it was trying to connect to us-west-2.queue.amazonaws.com. But i created the queue under sqs.us-west-2.amazonaws.com. May be they are redirected to the same host.

So what am i doing wrong? Any idea?

Thanks in advance.

1
amazon-web-servicesaws-sdkamazon-sqsaws-cli
us-west-2.queue... and sqs.us-west-2... should be functionally equivalent. It sounds as if the credentials in question lack sufficient permissions. – Michael - sqlbot
I had attached AmazonSQSFullAccess policy to my user but in IAM Management Console->Policies->AmazonSQSFullAccess Description it tells like that "Provides full access to Amazon SQS via the AWS Management Console.". Does it means that this policy does not provide access neither via aws cli nor aws sdk? – Ercan Celik
No, that policy should be good for any access method. Verify that you can use those credentials on aws-cli for other things? aws ec2 describe-availability-zones or aws iam get-user for example? – Michael - sqlbot
If you are pretty pretty sure you didn't make any mistake on the Accesskey ID and secret key, then try use policy simulator to validate the user really has the granted rights policysim.aws.amazon.com/home/index.jsp?# – mootmoot
I could list users under our account by "aws iam list-users" command. "aws iam" related commands are running successfully. But "aws sqs" related commands are failing – Ercan Celik

1 Answers

1
votes

The cause of access denied exception is that ForceMFA is used in every aws service except IAM. It was the reason that i could use IAM commands without problem.

So here are the steps :

  1. I created temporary access keys with the command below

    $ aws sts get-session-token --serial-number arn-of-the-mfa-device --token-code code-from-token

  2. Then i put the output values to my credentials file in a separate profile(e.g. mfa)

    aws_access_key_id = "Access-key-as-in-returned-output"

    aws_secret_access_key = "Secret-access-key-as-in-returned-output"

    aws_session_token = "Session-Token-as-in-returned-output"

  3. Then i called the queue list command like this :

    aws sqs list-queues --profile mfa