cloudqubes

Content outline

Apr 11, 2024
15 Min read

How to serve image files with Rails and Amazon S3

In this tutorial, we will build a Ruby on Rails app that serves image files using Amazon S3 as the storage service.

Cover image. HTTP request and response between a user and a Rails app.

Prerequisites

You can do this on a Linux, Mac, or Windows workstation.

You need an AWS account with privileges to create S3 buckets and security credentials (Access Keys).

We are using Rails version 7 in this tutorial. We assume that you know the basics of Ruby on Rails.

You must also be familiar with using web developer tools in the browser to inspect elements in a web page.

Outline

Background

Ruby on Rails is a feature-rich framework for building web applications.

Rails implements file uploading capability in Active Storage which is a built-in component in the Rails framework. Active Storage uses storage services like Amazon S3 for storing and serving files.

Active Storage can serve stored files in three different modes.

  1. Redirect mode
  2. Proxy mode
  3. Public mode

Redirect mode

Redirect mode serves stored files by redirecting each HTTP request to a temporary URL.

Rails Active Storage redirect mode By default, Active Storage stores files in S3 as private objects that cannot be accessed without authorization.

When the Rails app receives an HTTP request for a stored file, Active Record first requests S3 to create a presigned URL for the corresponding object. Then, Rails responds to the HTTP request with a 302 redirect response with the redirect URL set to the presigned URL. The web browser (or the client) then downloads the image from the presigned URL. The presigned URL is a temporary URL and expires in 300 seconds.

In the redirect mode, our Rails app does not have to bear the burden of serving the stored file to the user.

Proxy mode

Proxy mode serves files directly without HTTP redirect.

Rails Active Storage proxy mode

When an HTTP request is received, Active Storage first downloads the file from the storage and sends the file to the user in the HTTP response. The Proxy mode can quickly overload your Rails app if you are serving large files. So you must use a caching service between the user and the Rails app to offload some of the file serving workload to the caching service.

Public mode

Public mode is a variant of the redirect mode where Active Storage will redirect to a permanent URL instead of a temporary URL. In the public mode the S3 bucket must store files as public objects.

Rails Active Storage proxy mode

Building the Rails application

We will build a bare-bones Rails application with a Picture model that can store an image. We will not be building user authentication etc., in this tutorial. But you should be able to add those functions easily on top of the rails app that we develop here.

Install Ruby by following the installation instructions for your platform. If you are on Windows, we recommend using WSL to install Ruby.

Initialize a new Rails app

Install Rails.

$ gem install rails

Create a new Rails application.

$ rails new my-pictures
$ cd my-pictures

Run our new app.

$ bin/rails server

Go to http://127.0.0.1:3000 in your browser and you will get this default home page of the Rails app. Rails default home page.

Keep the rails server running and open a new tab to run the rest of the commands.

Create the Picture model (and view and the controller in one go)

Use scaffold to create the model, view, and controller code.

$ bin/rails g scaffold pictures title:string

This command creates the Picture model with an attribute title of type string along with the code for the view and the controller.

Run the database migration.

$ bin/rails db:migrate

Edit config/routes.rb to set the root URL to the index page of the Pictures controller.

  # config/routes.rb
  root "pictures#index"

Check that the rails server is still running and go to the home page http://127.0.0.1:3000. You’ll get the index page of the Pictures controller.

Updated home page

Enable Active Storage

Before writing code to store and serve the image files, we must enable Active Storage in our Rails app.

$ bin/rails active_storage:install
$ bin/rails db:migrate

Active storage depends on image_processing gem so we must install it.

Uncomment this line in the Gemfile.

gem "image_processing", "~> 1.2"

Install the gem.

$ bundle install

Upload and serve the image with Active Storage

We are going to write code to upload and serve an image file.

Update the Picture model to define the attribute filename which refers to a Active Storage file.

# app/models/picture.rb

class Picture < ApplicationRecord
  has_one_attached :filename
end

Update the picture_params private method in the picture_controller.rb to parse the filename parameter when creating a new instance of the Picture model.

# app/controllers/picture_controller.rb

class PicturesController < ApplicationController
.
.
.
  private
    def picture_params
      params.require(:picture).permit(:title, :filename)
    end
.
.
.
end

Add file attach capability in the _form.html.erb partial of the pictures view.

# app/views/pictures/_form.html.erb

<%= form_with(model: picture) do |form| %>
.
.
.
  <div>
    <%= form.label :filename, style: "display: block" %>
    <%= form.file_field :filename %>
  </div>
.
.
. 
<% end %>

Update the _picture.html.erb partial in the picture view to display the image.

<div id="<%= dom_id picture %>">
.
.
.
  <div>
    <% if picture.filename.attached? %>
      <%= image_tag picture.filename %>
    <%  end %>
  </div>
.
.
.
</div>

Run the rails server if it’s not already running and go to http://127.0.0.1:3000 in the browser.

Click on New Picure. Enter my bike for the title. Choose a picture of your bike and click on the Create picture button. (That’s if you are a 🚲 lover, if not choose whatever title and picture you want.)

You’ll be redirected to http://127.0.0.1:3000/picture/1 and the image that you just uploaded will be displayed.

This file is stored and served from the disk-based local storage in Active Record.

Configuring Active Storage to use Amazon S3

Let’s configure Active Storage to use Amazon S3 to store uploaded files.

Add the aws-sdk-s3 gem to the Gemfile.

gem "aws-sdk-s3", require: false

Install the gem.

$ bundle install

In the config/storage.yml file, add S3 bucket configuration.

# config/storage.yml

amazon:
  service: S3
  access_key_id: <%= Rails.application.credentials.dig(:aws, :access_key_id) %>
  secret_access_key: <%= Rails.application.credentials.dig(:aws, :secret_access_key) %>
  region: us-east-1
  bucket: cloudqubes-my-pictures

An S3 bucket name must be globally unique. If you try to create a bucket with a name that’s already in use, you’ll get an error. I will be using cloudqubes-my-pictures as the bucket name. Choose a different unique name for your S3 bucket.

Create new S3 bucket

Log in to the AWS GUI console and select S3 from the Services menu.

Click on Create bucket. Type in a bucket name. This must be the name of the bucket you configure in config/storage.yml

An S3 bucket name must be globally unique.

Leave the default values for all other parameters and click Create bucket at the bottom.

Create AWS access credentials.

We must provide Active Storage with AWS Access Keys to upload files to the S3 bucket via AWS SDK.

Log in to your AWS account. Click on your username in the top right and click on Security credentials.

If you do not see the menu item Security credentials, your account does not have the privileges to create credentials.

In the Access keys section click on Create access key.

Select option Other for the Usecase.

Type in ror-app for Description tag value and click on Create access key.

Copy the values Access key and Secret access key. Click on Done button.

You cannot retrieve the Secret access key again from the AWS GUI. If you lost your access key, you must create a new access key.

Configure the access keys in the Rails app.

The storage.yml file refers to Rails encrypted credentials for the AWS Access Keys.

So, let’s edit the Rails credentials.

$ EDITOR="vi" rails credentials:edit

This command decrypts and opens the file config/credentials.yml.enc in the editor vi. This file is intended to store all credentials used by our Rails app.

Insert these lines with the created AWS access keys.

aws:
  access_key_id: xxx
  secret_access_key: yyyy

Configure Rails to use the S3 bucket in the development mode

In the development mode, Rails uses the local storage as configured by this line in config/environment/development.rb

#config/environment/development.rb

  config.active_storage.service = :local

Let’s change the value to use Amazon S3.

#config/environment/development.rb

  config.active_storage.service = :amazon

:amazon refers to the configuration block we added in config/storage.yml.

Now that we have configured Rails to use the Amazon S3 in the development environment, let’s delete the previously created pictures from the Rails console.

$ bin/rails console
irb(main):001> Picture.delete_all
irb(main):001> exit

Serving files from Amazon S3 in Active Storage redirect mode

Active Storage serves the stored files in redirect mode by default. Let’s upload a new picture and confirm that it’s served by S3 in the redirect mode.

Run the Rails app if it’s not already running.

$ bin/rails server

Go to http://127.0.0.1 in the browser and create a new picture as we did before. If everything works fine, the file will be uploaded to the S3 bucket and will be served from there.

Open the web developer tools in the browser and inspect the URL of the image displayed in the browser.

<img src="http://127.0.0.1:3000/rails/active_storage/blobs/redirect/eyJfcmFpbHMiOnsiZGF0YSI6MywicHVyIjoiYmxvYl9pZCJ9fQ==--139946a806c262721fc0285c2c6de5e46b4079e6/terminal.png">

Check the URL with curl. Use the -v option so that curl will print the headers also.

$ curl http://127.0.0.1:3000/rails/active_storage/blobs/redirect/eyJfcmFpbHMiOnsiZGF0YSI6MywicHVyIjoiYmxvYl9pZCJ9fQ==--139946a806c262721fc0285c2c6de5e46b4079e6/terminal.png -v
*   Trying 127.0.0.1:3000...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 3000 (#0)
> GET /rails/active_storage/blobs/redirect/eyJfcmFpbHMiOnsiZGF0YSI6MywicHVyIjoiYmxvYl9pZCJ9fQ==--139946a806c262721fc0285c2c6de5e46b4079e6/terminal.png HTTP/1.1
> Host: 127.0.0.1:3000
> User-Agent: curl/7.68.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 302 Found
< x-frame-options: SAMEORIGIN
< x-xss-protection: 0
< x-content-type-options: nosniff
< x-permitted-cross-domain-policies: none
< referrer-policy: strict-origin-when-cross-origin
< date: Tue, 09 Apr 2024 09:22:41 GMT
< location: https://cloudqubes-my-pictures.s3.amazonaws.com/seosq4jzy2dtxno2gyxsjqoo9ao4?response-content-disposition=inline%3B%20filename%3D%22terminal.png%22%3B%20filename%2A%3DUTF-8%27%27terminal.png&response-content-type=image%2Fpng&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIASHBRS5NI62Z2NOLD%2F20240409%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240409T092241Z&X-Amz-Expires=300&X-Amz-SignedHeaders=host&X-Amz-Signature=2198c22e177daea4ed1e0d45acdfdb7d80afcef8bb00c19cdc8815b9a2761a79
< content-type: text/html; charset=utf-8
< cache-control: max-age=300, private
< x-request-id: efdf9189-35f4-468b-9625-415b646a0609
< x-runtime: 0.010782
< server-timing: start_processing.action_controller;dur=0.01, sql.active_record;dur=0.18, instantiation.active_record;dur=0.15, service_url.active_storage;dur=1.42, redirect_to.action_controller;dur=0.08, process_action.action_controller;dur=6.15
< Content-Length: 0

Rails redirect the request to the temporary URL indicated by the location. This URL points to our S3 bucket. Open this URL in a new browser tab and the image will be loaded in the browser window.

This URL expires in 300 seconds. Wait 5+ minutes and try to go to the URL again from the browser. You’ll get an Access Denied response.

<Error>
<Code>AccessDenied</Code>
<Message>Request has expired</Message>
<X-Amz-Expires>300</X-Amz-Expires>
<Expires>2024-04-09T09:27:41Z</Expires>
<ServerTime>2024-04-09T09:29:05Z</ServerTime>
<RequestId>V5RKZ5XAY0EBRECM</RequestId>
<HostId>PXEnGHDl1H507AOOooqvfB/aEOUk3hUOlOz/nUcY8eJW0h92mwmx+U4UMjiazmAKOYZevHz4oIo+3ktezqrrml3oFE2WcrdD</HostId>
</Error>

Serving files from Amazon S3 in Active Storage proxy mode

Now, we will update the code to serve the image in proxy mode.

Update the image_tag in _picture.html.erb.

  # app/views/pictures/_picture.html.erb
  
  <div>
    <% if picture.filename.attached? %>
      <%= image_tag rails_storage_proxy_path picture.filename %>
    <%  end %>
  </div>

Refresh the browser and inspect the image tag with the browser web developer tools.

<img src="/rails/active_storage/blobs/proxy/eyJfcmFpbHMiOnsiZGF0YSI6MywicHVyIjoiYmxvYl9pZCJ9fQ==--139946a806c262721fc0285c2c6de5e46b4079e6/terminal.png">

Use curl to affirm that there’s no redirect.

$ curl <image_url> -v

If you want to make the proxy mode the default file serving method, set this in config/initializers/active_storage.rb.

# config/initializers/active_storage.rb

Rails.application.config.active_storage.resolve_model_to_route = :rails_storage_proxy

Serving files from S3 in Active Storage public mode

In the public mode, Active Storage will redirect the HTTP request to the S3 object’s actual URL instead of generating a temporary URL. So, the S3 bucket and the objects must be publicly accessible in this mode.

Create new public S3 bucket

Open AWS S3 GUI console. Click on Create bucket.

Type in the name as cloudqubes-my-pictures-public. In the Object ownership section select ACL enabled. In the Block public access settings unselect Block all public access.

Creating public S3 bucket

Click on ‘Create bucket`.

Now, we have an S3 bucket that is publicly accessible. Let’s update the bucket name in storage.yml.

#config/storage.yml
amazon:
  service: S3
  access_key_id: <%= Rails.application.credentials.dig(:aws, :access_key_id) %>
  secret_access_key: <%= Rails.application.credentials.dig(:aws, :secret_access_key) %>
  region: us-east-1
  bucket: cloudqubes-my-pictures-public
  public: true

We also add directive public: true to enable public mode.

Delete all the previously created pictures.

$ bin/rails console
irb(main):001> Picture.delete_all
irb(main):001> exit

Run the rails app and create a new picture as we did before. Inspect the new image with the browser web developer tools.

<img src="http://127.0.0.1:3000/rails/active_storage/blobs/redirect/eyJfcmFpbHMiOnsiZGF0YSI6MywicHVyIjoiYmxvYl9pZCJ9fQ==--49d87fc65ddbc40ecb6c04d3c081bfe4589db178/alr5-1.webp">

Check the redirect URL with curl.

$ curl http://127.0.0.1:3000/rails/active_storage/blobs/redirect/eyJfcmFpbHMiOnsiZGF0YSI6MywicHVyIjoiYmxvYl9pZCJ9fQ==--49d87fc65ddbc40ecb6c04d3c081bfe4589db178/alr5-1.webp -v
*   Trying 127.0.0.1:3000...
* Connected to 127.0.0.1 (127.0.0.1) port 3000
> GET /rails/active_storage/blobs/redirect/eyJfcmFpbHMiOnsiZGF0YSI6MywicHVyIjoiYmxvYl9pZCJ9fQ==--49d87fc65ddbc40ecb6c04d3c081bfe4589db178/alr5-1.webp HTTP/1.1
> Host: 127.0.0.1:3000
> User-Agent: curl/8.4.0
> Accept: */*
> 
< HTTP/1.1 302 Found
< x-frame-options: SAMEORIGIN
< x-xss-protection: 0
< x-content-type-options: nosniff
< x-permitted-cross-domain-policies: none
< referrer-policy: strict-origin-when-cross-origin
< date: Wed, 10 Apr 2024 07:51:52 GMT
< location: https://cloudqubes-my-pictures-2.s3.amazonaws.com/p2s88v7j8zn27cf0s36375pp98ag
< content-type: text/html; charset=utf-8
< cache-control: max-age=300, private
< x-request-id: aedd7b4e-40f5-4ece-bef0-c28b1cad0b5c
< x-runtime: 0.050841
< server-timing: start_processing.action_controller;dur=0.01, sql.active_record;dur=24.32, instantiation.active_record;dur=0.38, service_url.active_storage;dur=0.68, redirect_to.action_controller;dur=0.05, process_action.action_controller;dur=28.82
< Content-Length: 0
< 
* Connection #0 to host 127.0.0.1 left intact

We can see that the redirected URL is the permanent URL of the image file object stored in our S3 bucket.

https://cloudqubes-my-pictures-2.s3.amazonaws.com/p2s88v7j8zn27cf0s36375pp98ag

Selecting a file serving mode for your Rails app

OK, now that we know how Rails serves image files, we can choose the best method for our next Rails app.

The redirect mode is suitable for most use cases. You can keep your S3 bucket private and Rails will create temporary URLs that expire in five minutes. Using this method, you can also allow only authorized users to access the files.

Choose the proxy mode if you intend to use a caching layer between your app and the users. But, every cache miss will hit your Rails app so you have to dimension your cloud infra accordingly.

Use the public mode only if you want to allow access to your files from clients other than your Rails application. In this mode, your S3 objects would be accessible to anyone on the Internet.

Wrapping up

Now, we have a Rails app that can store and serve images from Amazon S3.

In upcoming tutorials, we will build more interesting stuff with Rails and Amazon web services.