How to retrieve attachment url with Rails Active Storage with S3

Question

rails version 5.2

I have a scenario where I need to access the public URL of Rails Active Storage with Amazon S3 storage to make a zip file with Sidekiq background job.

I am having difficulty getting the actual file URL. I have tried rails_blob_url but it gives me following

http://localhost:3000/rails/active_storage/blobs/eyJfcmFpbHMiOnsibWVzc2FnZSI6IkJBaHBBZUk9IiwiZXhwIjpudWxsLCJwdXIiOiJibG9iX2lkIn19--9598613be650942d1ee4382a44dad679a80d2d3b/sample.pdf

How do I access the real file URL through Sidekiq?

storage.yml

test:
  service: Disk
  root: <%= Rails.root.join("tmp/storage") %>

local:
  service: Disk
  root: <%= Rails.root.join("storage") %>

development:
  service: S3
  access_key_id: 'xxxxx'
  secret_access_key: 'xxxxx'
  region: 'xxxxx'
  bucket: 'xxxxx'

development.rb

  config.active_storage.service = :development

I can access fine these on web interface but not within Sidekiq

Answer 1

Use ActiveStorage::Blob#service_url. For example, assuming a Post model with a single attached header_image:

@post.header_image.service_url

---

Update: Rails 6.1

Since Rails 6.1 ActiveStorage::Blob#service_url is deprecated in favor of ActiveStorage::Blob#url.

So, now

@post.header_image.url

is the way to go.

Sources:

• Link to the corresponding PR.
• Link to source.

Answer 2

My use case was to upload images to S3 which would have public access for ALL images in the bucket so a job could pick them up later, regardless of request origin or URL expiry. This is how I did it. (Rails 5.2.2)

First, the default for new S3 bucked is to keep everything private, so to defeat that there are 2 steps.

1. Add a wildcard bucket policy. In AWS S3 >> your bucket >> Permissions >> Bucket Policy

{
    "Version": "2008-10-17",
    "Statement": [
        {
            "Sid": "AllowPublicRead",
            "Effect": "Allow",
            "Principal": "*",
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::your-bucket-name/*"
        }
    ]
}

2. In your bucket >> Permissions >> Public Access Settings, be sure Block public and cross-account access if bucket has public policies is set to false

Now you can access anything in your S3 bucket with just the blob.key in the url. No more need for tokens with expiry.

Second, to generate that URL you can either use the solution by @Christian_Butzke: @post.header_image.service.send(:object_for, @post.header_image.key).public_url

However, know that object_for is a private method on service, and if called with public_send would give you an error. So, another alternative is to use the service_url per @George_Claghorn and just remove any params with a url&.split("?")&.first. As noted, this may fail in localhost with a host missing error.

Here is my solution or an uploadable "logo" stored on S3 and made public by default:

#/models/company.rb
has_one_attached :logo
def public_logo_url
    if self.logo&.attachment
        if Rails.env.development?
            self.logo_url = Rails.application.routes.url_helpers.rails_blob_url(self.logo, only_path: true)
        else
            self.logo_url = self.logo&.service_url&.split("?")&.first
        end
    end
    #set a default lazily
    self.logo_url ||= ActionController::Base.helpers.asset_path("default_company_icon.png")
end

Enjoy ^_^

Answer 3

If you need all your files public then you must make public your uploads:

In file config/storage.yml

amazon:
  service: S3
  access_key_id: zzz
  secret_access_key: zzz
  region: zzz
  bucket: zzz
  upload:
    acl: "public-read"

In the code

attachment = ActiveStorage::Attachment.find(90)
attachment.blob.service_url # returns large URI
attachment.blob.service_url.sub(/\?.*/, '') # remove query params

It will return something like: "https://foo.s3.amazonaws.com/bar/buz/2yoQMbt4NvY3gXb5x1YcHpRa"

It is public readable because of the config above.

Answer 4

Using the service_url method combined with striping the params to get a public URL was good idea, thanks @genkilabs and @Aivils_Štoss!

There is however a potential scaling issue involved if you are using this method on large number of files, eg. if you are showing a list of records that have files attached. For each call to service_url you will in your logs see something like:

DEBUG -- : [8df9220c-e8c9-45b7-a1ee-b746e623ca1b]   S3 Storage (1.4ms) Generated URL for file at key: ...

You can't eager load these calls either, so you can potentially have a large number of calls to S3 Storage to generate those URLs for each record you are showing.

I worked around it by creating a Presenter like this:

class FilePresenter < SimpleDelegator
  def initialize(obj)
    super
  end

  def public_url
    return dev_url if Rails.env.development? || Rails.env.test? || assest_host.nil?

    "#{assest_host}/#{key}"
  end

  private

  def dev_url
    Rails.application.routes.url_helpers.rails_blob_url(self, only_path: true)
  end

  def assest_host
    @assest_host ||= ENV['ASSET_HOST']
  end
end

Then I set an ENV variable ASSET_HOST with this:

https://<your_app_bucket>.s3.<your_region>.amazonaws.com

Then when I display the image or just the file link, I do this:

<%= link_to(image_tag(company.display_logo),
    FilePresenter.new(company.logo).public_url, target: "_blank", rel:"noopener") %>

<a href=<%= FilePresenter.new(my_record.file).public_url %> 
   target="_blank" rel="noopener"><%= my_record.file.filename %></a>

Note, you still need to use display_logo for images so that it will access the variant if you are using them.

Also, this is all based on setting my AWS bucket public as per @genkilabs step #2 above, and adding the upload: acl: "public-read" setting to my 'config/storage.yml' as per @Aivils_Štoss!'s suggestion.

If anyone sees any issues or pitfalls with this approach, please let me know! This seemed to work great for me in allowing me to display a public URL but not needing to hit the S3 Storage for each record to generate that URL.

Answer 5

A bit late, but you can get the public URL also like this (assuming a Post model with a single attached header_image as in the example above):

@post.header_image.service.send(:object_for, @post.header_image.key).public_url

Update 2020-04-06

1. You need to make sure, that the document is saved with public ACLs (e.g. setting the default to public)

2. rails_blob_url is also usable. Requests will be served by rails, however, those requests will be probably quite slow, since a private URL needs to be generated on each request. (FYI: outside the controller you can generate that URL also like this: Rails.application.routes.url_helpers.rails_blob_url(@post, only_path: true))