MalformedPolicy error using aws s3api

9
votes

I am trying to use put-bucket-policy to add a policy to an s3 bucket via the aws s3api (Windows).

I am using precisely the policy code given here http://docs.aws.amazon.com/AmazonS3/latest/dev/AccessPolicyLanguage_UseCases_s3_a.html under "Granting Permission to an Anonymous User" with my bucket name substituted in.

I am getting

A client error (MalformedPolicy) occured: policies must be valid JSON and the first byte must be '{'

Any clues?

EDIT: Inlining the JSON works- so it is some kind of file format issue - just not one that I can see. Be great to be able to get it working with files.

EDIT: To help anyone else who maybe ends up here, buckets should be named all in lowercase. If not, some s3/s3api operations work, but not others. See here

3
amazon-web-servicesamazon-s3
Sounds like you're not. :) Try validating your policy document? jsonlint.com Also seems possible that if your JSON is valid, then the editor you're using is attaching some noise, like a utf BOM to the beginning of the file. – Michael - sqlbot
Already checked on JSONLint - it's valid. Have looked in a hex editor, and cropped an initial coupla bytes, tried making the file UTF-8, Unicode, ANSI. Still the same error. – kpollock
When I have to use windows, I use Notepad++ to edit text, html, perl, json, sql, and no issues when I have to push the files back over to other systems... maybe give that a try? – Michael - sqlbot
I maybe will - it's actually okay inline for this job. – kpollock

3 Answers

14
votes

TL;DR


You must add file:// protocol scheme to the policy file path

$ aws s3api put-bucket-policy --bucket kryptonite \
   --policy file:///home/superman/aws-example/public-bucket-policy.json


Windows


$ aws s3api put-bucket-policy --bucket kryptonite \
   --policy file://C:\Temp\public-bucket-policy.json


relative path


$ aws s3api put-bucket-policy --bucket kryptonite \
   --policy file://public-bucket-policy.json


FULL STORY: How to set public bucket policy via CLI

First let's make sure there's no other policy on the bucket:

$ s3api get-bucket-policy --bucket kryptonite

A client error (NoSuchBucketPolicy) occurred when calling the   GetBucketPolicy
   operation: The bucket policy does not exist


Now let's make sure we have policy file in current directory and it contains valid json (mind name of the kryptonite bucket

$ ls
public-bucket-policy.json

$ cat public-bucket-policy.json 
{
  "Statement": [
    {
      "Resource": "arn:aws:s3:::kryptonite/*",
      "Action": "s3:GetObject",
      "Principal": "*",
      "Effect": "Allow",
      "Sid": "AddPerm"
    }
  ],
  "Version": "2012-10-17"
}


Now let's try to put the policy by specifying just filename 

$ s3api put-bucket-policy --bucket kryptonite --policy public-bucket-policy.json

A client error (MalformedPolicy) occurred when calling the PutBucketPolicy 
   operation: Policies must be valid JSON and the first byte must be '{'


Now let's make another attempt and specify the full path $ s3api put-bucket-policy --bucket kryptonite \ --policy /home/superman/aws-example/public-bucket-policy.json

A client error (MalformedPolicy) occurred when calling the PutBucketPolicy 
   operation: Policies must be valid JSON and the first byte must be '{'


Now let's add file:// prefix and it will work 

$ s3api put-bucket-policy --bucket kryptonite \
    --policy file:///home/superman/aws-example/public-bucket-policy.json


And we can now verify that this policy had been applied

$ s3api get-bucket-policy --bucket kryptonite
{
    "Policy": "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Sid\":\"AddPerm\",\"Effect\":\"Allow\",\"Principal\":\"*\",\"Action\":\"s3:GetObject\",\"Resource\":\"arn:aws:s3:::kryptonite/*\"}]}"
}


And as special bonus let's pipe the policy through the jq utility (twice) to extract correct field and format JSON nicely 

$ s3api get-bucket-policy --bucket kryptonite | jq .Policy --raw-output | jq .
{
  "Statement": [
    {
      "Resource": "arn:aws:s3:::kryptonite/*",
      "Action": "s3:GetObject",
      "Principal": "*",
      "Effect": "Allow",
      "Sid": "AddPerm"
    }
  ],
  "Version": "2012-10-17"
}


And as you can see the policy is correct

0
votes

I have gone with inling the JSON.

Couple of maybe useful hints for those that end up here.

  1. Bucket names: Buckets should be named all in lowercase. If not, some s3/s3api operations work (mb, put-bucket-policy), but not others (put-bucket-website) See http://support.rightscale.com/09-Clouds/AWS/FAQs/FAQ_0094_-_What_are_valid_S3_bucket_names%3F

  2. If, like me, you start by using get-bucket-website on a bucket created through the S3 console to get example JSON for making a bucket a website, it may not work. get-bucket-website gives you back blank entries for any unset parameters (e.g. RedirectAllRequestsTo) - which gives errors if used in put-bucket-website... Just miss parameters out if you don't need them and that works - e.g.

    {"IndexDocument":{"Suffix":"index.html"}}

is the minimum. (remember to escape the quotes if inlining!)

0
votes

For a "full inline" syntax, use :

aws s3api put-bucket-policy --bucket MYBUCKETNAME --policy "{\"Version\":\"2008-10-17\", \"Statement\":[{\"Sid\":\"AllowPublicRead\",\"Effect\":\"Allow\",\"Principal\":{\"AWS\":\"*\"},\"Action\":\"s3:GetObject\",\"Resource\":\"arn:aws:s3:::MYBUCKETNAME/*\"}]}"