Read and write from/to S3 bucket using access points with boto3

Question

I have to access S3 bucket using access points with boto3.

I have created an access point with a policy to allow reading and writing (<access_point_arn> is my access point ARN):

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": "*",
            "Action": ["s3:GetObject", "s3:PutObject"],
            "Resource": "<access_point_arn>/object/*"
    ]
}

In the official documentation there is a mention about access points, where access point ARN has to come in place of bucket name (https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html). There are no examples on the official documentation site for developers (https://docs.aws.amazon.com/AmazonS3/latest/dev/using-access-points.html).

So based on the information I assume that the right way to use it is:

import boto3
s3 = boto3.resource('s3')
s3.Bucket('<access_point_arn>').download_file('hello.txt', '/tmp/hello.txt')

When I execute this code in Lambda with AmazonS3FullAccess managed policy attached I am getting an ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden

Both Lambda and S3 access point are connected to the same VPC.

Is your AWS Lambda function connected to a VPC? Has the Access Point been connect to the Internet, or to the VPC? Does the IAM Role assigned to the Lambda function have permission to access the bucket (can you show us the policy)? Please Edit your question to add these details, so we can attempt to reproduce the situation. — John Rotenstein
Thanks for reaching out @JohnRotenstein! I have updated the description - both Lambda and S3 access point are connected to the same VPC. I have also updated the policy code, as the one I am using has the following entry for Resource - "Resource": "<access_point_arn>/object/*". Lambda has the managed AmazonS3FullAccess policy attached which I assumed would give it all the permissions necessary. — Krzysztof Słowiński

KoppiaPaste KoppiaPaste · Accepted Answer · 2020-07-08T16:45:13

My first guess is that you are missing permissions that have to be defined (1) on the bucket (bucket policy) and (2) on the IAM user or role which you are using in the boto3 SDK.

(1) From the documentation I can see that

For an application or user to be able to access objects through an access point, both the access point and the underlying bucket must permit the request.

You could, for instance, add a bucket policy that is delegating access control to access points so that you don't have to specify each principal that comes via the access points. An example is given in the linked docs.

(2) As stated in your question, you are already using AmazonS3FullAccess policy in your LambdaExecutionRole. My only guess (i.e. what happened to me) is that there is, e.g., KMS encryption on the objects in your bucket and your role is missing permissions for kms actions. Try executing the function with Admin policy attached and see if it works. If it does, find out which specific permissions are missing.

Some further notes: I assume you

didn't restrict the access point to be available within a specific VPC only.
are blocking public access.

Read and write from/to S3 bucket using access points with boto3

2 Answers