Rails ActiveStorage uploads work locally but AWS & GCS uploads fail with 500 errors

Question

I've just finished implementing ActiveStorage uploads to my Rails 6 app.

I'm uploading an image when I create an Organisation. The organisation gets created, and the image gets added to the two tables - active_storage_attachments and active_storage_blobs.

My application logs then say the image was uploaded to Google Cloud Storage (or AWS), and displays a 500 Server Error followed by a System Stack error.

17:06:30 web.1     |    (0.2ms)  BEGIN
17:06:30 web.1     |   ↳ app/controllers/organisations_controller.rb:45:in `update'
17:06:30 web.1     |   ActiveStorage::Blob Load (0.8ms)  SELECT "active_storage_blobs".* FROM "active_storage_blobs" INNER JOIN "active_storage_attachments" ON "active_storage_blobs"."id" = "active_storage_attachments"."blob_id" WHERE "active_storage_attachments"."record_id" = $1 AND "active_storage_attachments"."record_type" = $2 AND "active_storage_attachments"."name" = $3 LIMIT $4  [["record_id", 2], ["record_type", "Organisation"], ["name", "image"], ["LIMIT", 1]]
17:06:30 web.1     |   ↳ app/controllers/organisations_controller.rb:45:in `update'
17:06:30 web.1     |   ActiveStorage::Attachment Load (0.3ms)  SELECT "active_storage_attachments".* FROM "active_storage_attachments" WHERE "active_storage_attachments"."record_id" = $1 AND "active_storage_attachments"."record_type" = $2 AND "active_storage_attachments"."name" = $3 LIMIT $4  [["record_id", 2], ["record_type", "Organisation"], ["name", "image"], ["LIMIT", 1]]
17:06:30 web.1     |   ↳ app/controllers/organisations_controller.rb:45:in `update'
17:06:30 web.1     |   ActiveStorage::Blob Create (40.9ms)  INSERT INTO "active_storage_blobs" ("key", "filename", "content_type", "metadata", "byte_size", "checksum", "created_at") VALUES ($1, $2, $3, $4, $5, $6, $7) RETURNING "id"  [["key", "4e1y1yhbggpS6ncqnqd0j8y0knvxkx"], ["filename", "logo-icon.png"], ["content_type", "image/png"], ["metadata", "{\"identified\":true}"], ["byte_size", 2055], ["checksum", "VT91aKRpVPAZnzNeOLdeQQ=="], ["created_at", "2021-03-21 17:06:30.892165"]]
17:06:30 web.1     |   ↳ app/controllers/organisations_controller.rb:45:in `update'
17:06:30 web.1     |   ActiveStorage::Attachment Create (3.6ms)  INSERT INTO "active_storage_attachments" ("name", "record_type", "record_id", "blob_id", "created_at") VALUES ($1, $2, $3, $4, $5) RETURNING "id"  [["name", "image"], ["record_type", "Organisation"], ["record_id", 2], ["blob_id", 20], ["created_at", "2021-03-21 17:06:30.940291"]]
17:06:30 web.1     |   ↳ app/controllers/organisations_controller.rb:45:in `update'
17:06:30 web.1     |    (0.5ms)  COMMIT
17:06:30 web.1     |   ↳ app/controllers/organisations_controller.rb:45:in `update'
17:06:31 web.1     |   S3 Storage (241.6ms) Uploaded file to key: 4e1y1yhbggp6ncqnq0j8y0knvxkx (checksum: VT91aKRpVPAZnzNeOLdeQQ==)
17:06:31 web.1     | Completed 500 Internal Server Error in 408ms (ActiveRecord: 58.9ms | Allocations: 105587)
17:06:31 web.1     | 
17:06:31 web.1     | 
17:06:31 web.1     |   User Load (0.7ms)  SELECT "users".* FROM "users" WHERE "users"."id" = $1 ORDER BY "users"."id" ASC LIMIT $2  [["id", 1], ["LIMIT", 1]]
17:06:31 web.1     |   
17:06:31 web.1     | SystemStackError (stack level too deep):
17:06:31 web.1     |   
17:06:31 web.1     | app/controllers/organisations_controller.rb:45:in `update'

My application then hangs and I need to restart it. I check the Google Cloud bucket, and the image hasn't been added. I've tried the exact same steps with AWS and the same error occurs.

Does anyone know what's going on here? When I use local disk storage, everything works as expected.

Here are my configs:-

Gemfile

gem 'rails', '~> 6.0.0.rc2'
gem 'aws-sdk-s3', require: false

config/storage.yml

amazon:
  service: S3
  access_key_id: <%= Rails.application.credentials.dig(:aws, :access_key_id) %>
  secret_access_key: <%= Rails.application.credentials.dig(:aws, :secret_access_key) %>
  region: eu-west-1
  bucket: app-local

config/environments/development.rb

  # Store uploaded files on the local file system (see config/storage.yml for options).
  config.active_storage.service = :amazon

models/organisation.rb

class Organisation < ApplicationRecord
  has_one_attached :image
end

controllers/organisations_controller.rb

  def update
    if @organisation.update(organisation_params)  ### This is line 45 which is erroring
      redirect_to @organisation, notice: 'Organisation was successfully updated.'
    else
      render :edit
    end
  end

views/organisations/_form.html.erb

  <div class="form-group">
    <%= form.label :image %>
    <%= form.file_field :image, class: 'form-control' %>
  </div>

Is there a way to get more log info from the AWS or GCS gems, or something I can try to diagnose this further? The 500 error isn't helpful.

s89_ s89_ · Accepted Answer · 2021-03-21T18:54:30

The answer turned out to be that I was getting "Access denied" from AWS - so my permissions in AWS were somehow set up wrongly.

I only learned this because I turned Access Logs on in AWS (off by default), so was able to at least diagnose it further than the 500 error.

Rails ActiveStorage uploads work locally but AWS & GCS uploads fail with 500 errors

1 Answers