Amazon S3 - Copy pasted file path issue

Question

I am trying to get all file paths in a bucket. Using listobjects function I can able to achieve it. But there is a file path issue.

Consider this is the folder structure

SampleBucket --> A (root folder) --> B (subfolder)--> c(subfolder within B)

What I need is

http://SampleBucket.s3.amazonaws.com/A/

http://SampleBucket.s3.amazonaws.com/A/B/

http://SampleBucket.s3.amazonaws.com/A/B/C

What I am getting is

http://SampleBucket.s3.amazonaws.com/A/

http://SampleBucket.s3.amazonaws.com/A/B/C

This happened when I created "A" folder & copy pasted "B" folder from my local drive to cloud berry

This is my code to get all folders & files in a bucket ..

public List<S3ObjectSummary> getObjectslistFromFolder(String bucketName, String folderKey) {    
AWSCredentials awsCreds = new BasicAWSCredentials(accessKey, secretKey);
AmazonS3 s3client = new AmazonS3Client(awsCreds); 
ObjectListing current = s3client.listObjects(bucketName,folderKey);
List<S3ObjectSummary> keyList = current.getObjectSummaries();
current = s3client.listNextBatchOfObjects(current);
while (current.isTruncated()){
    keyList.addAll(current.getObjectSummaries());
    current = s3client.listNextBatchOfObjects(current);
}
keyList.addAll(current.getObjectSummaries());   
return keyList;

}

Answer 1

S3 does not have folders. It basically has key => object mappings + the ability to match on the key prefix.

When you are looking for A/ you are looking at all the keys that have a prefix that starts with A/

The illusion of folders is usually built client-side by the software you are using (cloudberry) or if your application logic depends on this you can also parse out the keys that are coming back to "recreate" the folders locally.

Again, S3 only knows about keys and can do prefix matching on keys.

Answer 2

Seems cloud berry created two objects. a/ and a/b/c/. I could reproduce it with these cli commands:

$ aws s3api put-object --bucket testBucket --key "a/" 
$ aws s3api put-object --bucket testBucket --key "a/b/c/" 
$ aws s3api list-objects --bucket testBucket --prefix a --output json | jq -r .Contents[].Key
a/
a/b/c/

In AWS S3 console they look like normal subfolders, a, b and c. You should know that S3 doesnt really have subfolders, the / is just part of the keys.