How to delete multiple objects with the same key in aws s3 bucket

Question

TLDR; How can I delete image replicas in subfolders of s3 with the same key as the original image?

I've got a prisma server and upload images from my app to my s3 bucket through the prisma backend. Moreover, I run a lambda function to resize these images on-the-fly if requested.

Here's the process of the lambda function

A user requests a resized asset from an S3 bucket through its static website hosting endpoint. The bucket has a routing rule configured to redirect to the resize API any request for an object that cannot be found.
Because the resized asset does not exist in the bucket, the request is temporarily redirected to the resize API method.
The user’s browser follows the redirect and requests the resize operation via API Gateway.
The API Gateway method is configured to trigger a Lambda function to serve the request.
The Lambda function downloads the original image from the S3 bucket, resizes it, and uploads the resized image back into the bucket as the originally requested key.
When the Lambda function completes, API Gateway permanently redirects the user to the file stored in S3.
The user’s browser requests the now-available resized image from the S3 bucket.
Subsequent requests from this and other users will be served directly from S3 and bypass the resize operation.
If the resized image is deleted in the future, the above process repeats and the resized image is re-created and replaced into the S3 bucket.

https://aws.amazon.com/blogs/compute/resize-images-on-the-fly-with-amazon-s3-aws-lambda-and-amazon-api-gateway/

This brings me to the following issue: Whenever I delete an image-node with a key in Prisma I can delete the object with the same key from aws s3, yet I won't touch the resized replicas of it in the subfolders of the respective resolutions. How can I achieve this? I tried using aws' deleteObjects() by passing in only one key as shown below. However, this only deletes the original image at the root of the bucket.

Here's the lambda functions implementation

exports.processDelete = async ( { id, key }, ctx, info) => {

  const params = {
    Bucket: 'XY',
    Delete: {
      Objects: [
        {
          Key: key, 
        },
      ],
      Quiet: false
    }
  }

  // Delete from S3
  const response = await s3
    .deleteObjects(
      params,
      function(err, data) {
        if (err) console.log(err, err.stack); // an error occurred
        else     console.log(data);           // successful response
      }
    ).promise()

  // Delete from Prisma
  await ctx.db.mutation.deleteFile({ where: { id } }, info)

  console.log('Successfully deleted file!')
}

The object key is full/path/prefix/filename.ext. Deleting filename.ext deletes the file of that name from the root of the bucket. If you want to delete other files, you need to pass their entire object key, because by itself, filename.ext refers only to the object with exactly that key -- not any other similarly-named objects, because their object key is not actually "the same." Is that what you are asking? — Michael - sqlbot
@Michael-sqlbot I understand. I'm wondering how can implement somewhat of a cascading deletion of all resized images relating to the original file that is being deleted. — mowtheone

mowtheone mowtheone · Accepted Answer · 2018-05-08T13:18:23

Because I'm only allowing the resizing of certain resolutions, I ended up doing the following:

exports.processDelete = async ( { id, key }, ctx, info) => {
  const keys = [
    '200x200/' + key,
    '293x293/' + key,
    '300x300/' + key,
    '400x400/' + key,
    '500x500/' + key,
    '600x600/' + key,
    '700x700/' + key,
    '800x800/' + key,
    '900x900/' + key,
    '1000x1000' + key,
  ]

  const params = {
    Bucket: 'XY',
    Delete: {
      Objects: [
        {
          Key: key, 
        },
        {
          Key: keys[0], 
        },
        {
          Key: keys[1], 
        },
        {
          Key: keys[2], 
        },
        {
          Key: keys[3], 
        },
        {
          Key: keys[4], 
        },
        {
          Key: keys[5], 
        },
        {
          Key: keys[6], 
        },
        {
          Key: keys[7], 
        },
        {
          Key: keys[8], 
        },
        {
          Key: keys[9], 
        },
      ],
      Quiet: false
    }
  }

If there is a more elegant way, please let me know. :)

How to delete multiple objects with the same key in aws s3 bucket

TLDR; How can I delete image replicas in subfolders of s3 with the same key as the original image?

2 Answers