Error 403, Problem with importing image data in s3 to SageMaker Notebook

Question

I am working with SageMaker Notebook and image data in S3 bucket with name s3:///train/ and validate data in other dir. I create an IAM Role and put previous specific bucket, in the notebook I load this bucket with:

s3_train = 's3://<BucketName>/train'
train_data = sagemaker.session.s3_input(s3_train, distribution='FullyReplicated', 
                        content_type='application/x-image', s3_data_type='S3Prefix')

The same for train lst file, validation data and validation lst data, after create data channels with this:

data_channels = {'train': train_data, 'validation': validation_data, 
                 'train_lst': train_data_lst, 'validation_lst': validation_data_lst}

After create a TensorFlow estimator, and finally in fit pass the data with this:

tf_estimator.fit(inputs=data_channels, logs=True)

And return this menssage error: An error occurred (403) when calling the HeadObject operation: Forbidden

Answer 1

There are many reasons why this could occur but ultimately it comes down to permissions.

I recommend you check the following:

• Does your notebook role have permission to access the s3 object?
• Does the bucket have a bucket policy? Does it deny access?
• Does the object in S3 have an ACL in place?
• Does the object exist? This can trigger 403 if the object is not public

AWS have a more comprehensive list of checks available from their troubleshooting page.

Answer 2

You're seeing that error because the fit(...) method was unable to get access to your S3 bucket. You probably need to make changes to your IAM role you used in your notebook to allow access to your bucket.

SageMaker uses IAM roles to get access to your resources.

There is comprehensive documentation on how the various sub-features of SageMaker gain credentials to get access to user-resources along with the varied security-focused use-cases users may opt to use here: https://docs.aws.amazon.com/sagemaker/latest/dg/security-iam.html

Perhaps, if you have no special security requirements, then the easiest way to unblock yourself is to create an IAM role with the AmazonSageMakerFullAccess Policy attached to it and use it.

Alternatively, you could also use a pared down policy that should suffice to run a SageMaker Training job - either leave the "Resource": "*" bit untouched, or explicitly specify your S3 bucket.