Different permissions same S3 bucket, parquet files

0
votes

Problem I have multiple files in the same S3 bucket. When I try to load one file into Snowflake, I get a "access denied" error. When I try a different file (in the same bucket), I can successfully load into Snowflake.

The file highlighted does not load into Snowflake.

File does not load

This is the error

Error

Using a different file but in the same bucket, I can successfully load into Snowflake.

this file loads

Known Difference: The file that does not work was generated by AWS. The file that can be loaded into Snowflake was generated by AWS, saved to my local then reuploaded to the bucket.

The only difference is I brought it down to my local machine.

Question: Is there a known file permission on parquet files? Why does this behavior go away when I download and upload to the same bucket.

It cannot be an S3 bucket issue. It has to be some encoding on the parquet file.

2
amazon-web-servicesamazon-s3snowflake-cloud-data-platformparquet

2 Answers

1
votes

Question: Is there a known file permission on parquet files? Why does this behavior go away when I download and upload to the same bucket.

It cannot be an S3 bucket issue. It has to be some encoding on the parquet file.

You are making some bad assumptions here. Each S3 object can have separate ACL (permission) values. You need to check what the ACL settings are by drilling down to view the details of each of those objects in S3. My guess is AWS is writing the objects to S3 with a private ACL, and when you re-uploaded one of them to the bucket you saved it with a public ACL.

0
votes

Turns out I needed to add KMS permissions to the user accessing the file.