3
votes

I have a Heroku-hosted app that uses Paperclip to store User photos on Amazon S3

I want to move some (not all) files to a new bucket based on some internal logic (the app is multi-tenant and I'm separating AWS file storage and my Postgres DB into separate tenants/schemas)

enter image description here

I have 2 options I'm considering (drawn above)

Option 1 - Use the AWS Cli to move files directly between buckets

This option is AWS native, but it has the drawback of having to worry about an entire folder structure for each file (thumbnails, etc..). Moving a file involves moving all the various styles of the file - original, medium size, thumbnail, etc.. so it's not as straightforward as copying 1 file over.

It also copies everything over to the new bucket with the exact same folder/id structure, which I'd like to avoid since the User's corresponding DB info (e.g. id) will change when I migrate them over in the postgres DB

Option 2 - Use paperclip to pull down each file locally and re-upload it

This is an attractive option because it lets paperclip handle all the work.

However, paperclip uses the bucket name to construct the URL of the file. I need it to pull from 1 bucket and push to another bucket. Is there a way to set the bucket name individually for each transaction?

1

1 Answers

1
votes

Paperclip uses the bucket name to construct the URL of a remote file but the names of these directories and files doesn't depend on the bucket name. If your files or directories contains the bucket name then you are doing it wrong and you should start by fixing it.

Do the following:

  1. Sync your public/system directory with oldbucket using the aws s3 sync OLD_BUCKET_URL public/system command

  2. Perform the changes in the directories and files locally with a Ruby script using Paperclip

  3. Sync (upload) your public/system directory with newbucket using the aws s3 sync public/system NEW_BUCKET_URL command.