S3 List Objects request on a bucket to get additional keys (> 1000)

Question

By default, an S3 List Objects (Get Bucket) request returns 1000 keys. According to S3 documentation, if I want to list more objects, then I have to pass a marker query string along with my request with the value of the marker set to be the last key from the previous list of objects.

For some reason this doesn't work for me ... I seem to be missing something obvious and I'm trying to find out what.

This is what the REST request is supposed to look like:

GET ?marker=LAST-KEY HTTP/1.1
Host: quotes.s3.amazonaws.com
Date: Wed, 01 Mar  2009 12:00:00 GMT
Authorization: AWS ACCESS_KEY_ID:SIGNATURE

My PHP code looks something like this:

...
// this is where I'm getting the value of the last key
$marker=$arrObjects["Contents"][999]["Key"];
...
//this is where I'm signing my request
$canonicalizedResources = "/".$bucketName."/".(($marker=="")?"":"?marker=".rawurlencode($marker));
// $contentMD5 and $contentType are both empty strings ""
$stringToSign = utf8_encode("GET"."\n".$contentMD5."\n".$contentType."\n".$timestamp."\n".$canonicalizedAmzHeaders.$canonicalizedResources);
$signature = base64_encode(hash_hmac("sha1",$stringToSign,$customerSecretKey,true));
...

This code works perfectly if I'm listing the objects in a bucket which number less than 1000. But whenever I pass a marker to the request, I get a "Signature Does Not Match" error.

Anyone know where I'm going wrong?

Geoff Appleford Geoff Appleford · Accepted Answer · 2011-03-22T09:14:58

To follow up on @Skyler's comment, using the SDK your code could look like this (minus the authorization bits):

// Instantiate the class
$s3 = new AmazonS3();

$response = $s3->list_objects('my-bucket');

S3 List Objects request on a bucket to get additional keys (> 1000)

1 Answers