Elixir/Phoenix: Client-side / browser AJAX upload to S3

Question

My Elixir / Phoenix app needs to be able to record HTML5 video, allow the user to review their recorded video, then upload the recording directly to S3 for storage via AJAX and send a POST to the Elixir server to store the newly uploaded object. In the Rails world there's various gems and lengthy tutorials on how to accomplish this; what's a bare-bones way to do direct-to-S3 file uploads in Elixir / Phoenix?

Topher Hunt Topher Hunt · Accepted Answer · 2017-02-13T19:00:48

I eventually worked this out. There's setup required on the S3 account, server-side code to add, and client-side code to add.

S3 setup

First you need an AWS account and an S3 bucket. Create the bucket.

In the bucket settings -> Permissions -> Public access settings, ensure that "Block new public ACLs and uploading public objects" and "Remove public access granted through public ACLs" are not checked.

Then under Permissions -> CORS configuration, add a CORS config to allow cross-domain browser requests from your site, like this:

<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
  <CORSRule>
    <AllowedOrigin>http://localhost:4000</AllowedOrigin>
    <AllowedMethod>PUT</AllowedMethod>
    <AllowedHeader>*</AllowedHeader>
  </CORSRule>
</CORSConfiguration>

(Specify an <AllowedOrigin> statement for each origin where your site will be accessible, or set AllowedOrigin to * if you aren't concerned about CORS security.)

It's good hygiene to also set up an IAM role that only has access to this S3 bucket, and only the specific operations you want to allow. Then in the server setup step, you provide that IAM role's access credentials instead of your AWS account access credentials. I've skipped this step in the code shown here.

Server-side

In mix.exs, ensure the ex_aws and ex_aws_s3 deps are installed (note that the version is locked at 2.0):

  # in mix.exs, in the deps list
  # Stay at 2.0 to avoid presigned url bug: https://github.com/ex-aws/ex_aws/issues/602
  {:ex_aws, "2.0.1"},
  {:ex_aws_s3, "~> 2.0"},
  # required by :ex_aws
  {:sweet_xml, "~> 0.6"},

In config/config.exs, provide your AWS access credentials and S3 settings:

config :ex_aws,
  access_key_id:     System.get_env("AWS_ACCESS_KEY_ID"),
  secret_access_key: System.get_env("AWS_SECRET_ACCESS_KEY"),
  region: "us-east-1",
  s3: [
    scheme: "https://",
    host: "s3.amazonaws.com",
    region: "us-east-1" ]

(To be Heroku-compatible, I store sensitive values in environment variables set in config/secrets.exs, which is excluded from the Git repo.)

Now set up the controller action that renders the page on which the S3 file upload will occur. A minimal controller action might look like this:

def edit(conn, %{"id" => id}) do
  render conn, "edit.html", presigned_s3_url: presigned_s3_url(id)
end

defp presigned_s3_url(id) do
  config = ExAws.Config.new(:s3)
  bucket = System.get_env("S3_BUCKET")
  path = "uploads/interview_recordings/#{id}.webm"
  # Set the file permission so this file can be publicly linked
  query_params = [{"x-amz-acl", "public-read"}, {"contentType", "binary/octet-stream"}]
  # NOTE: Also set option `virtual_host: true` if using an EU region
  options = [query_params: query_params]
  {:ok, url} = ExAws.S3.presigned_url(config, :put, bucket, path, options)
  url
end

This #edit action provides a variable @presigned_s3_url which authorizes our client-side code to upload a file to S3. Note the :put method specified when calling ExAws.S3.presigned_url; if you specify a different request method or omit the query params etc., the S3 upload request will likely be rejected. The params you specify here must exactly match the params of the actual AJAX request, or else S3 will reject the upload because the signature S3 generates won't match the signature we generated.

Client-side

Your setup will vary, so I won't paste any HTML markup. The only important part is: some client action (in my case, recording an HTML5 video) causes a file to be available to the browser JS, and when the client clicks a "Submit" button, we make a PUT request to the signed S3 url with the file data attached. My AJAX call looks like this:

$('.js-submit-recording').click(function(e) {
  e.preventDefault();
  console.log("Uploading the recording to S3...");
  $.ajax({
    url: $(this).data('presigned-s3-url'), // the url we generated server-side
    type: "PUT",
    contentType: "binary/octet-stream",
    processData: false, // Treat the data as a raw file, not as a POST form
    data: recording_data, // a blob object
    success: function() {
      console.log("File successfully uploaded!");
      // Now you can initiate next steps like making a POST request to our
      // Phoenix server to record the newly uploaded file in our db
    },
    error: function() {
      console.log("File not uploaded :-/");
      console.log(arguments);
      // If you get errors, open Chrome's Network tab and look at the
      // response body for the S3 request. It will describe the reason / 
      // error and some details that can help with troubleshooting.
    }
  });
});

That should get the basics working!

Elixir/Phoenix: Client-side / browser AJAX upload to S3

2 Answers

S3 setup

Server-side

Client-side