How to use Solid Queue with Rails Active Job
This is the part 2 of a 3-part tutorial on sending emails with Rails.
- Part-1: How to send emails from Rails app via Google SMTP server
- Part-2: How to use Solid Queue with Rails Active Job (This tutorial)
- Part-3: How to scale Rails Solid Queue in production(to be completed)
In this tutorial, we will update the Rails app we created in part-1, to use Solid Queue as the Active Job backend.
If you have any other Rails 7 app using Active Job to run background tasks, you can follow the steps in this tutorial to add Solid Queue to your Rails app too.
Prerequisites
You can do this tutorial on a Linux, Mac, or Windows workstation.
Outline
- Install Solid Queue in an existing Rails app
- Configure Solid Queue as the Active Job backend
- Run Solid Queue in development mode
- Deploy our Rails application to production with Solid Queue
- Wrapping up
Install Solid Queue in an existing Rails app
At this point, you must have a working Rails app that uses Active Job to run any background task.
Add Solid Queue to the Gemfile in your Rails app:
gem 'solid_queue'
Install solid_queue
gem:
$ bundle install
This installs Solid Queue which is a database-based backend for Rails Active Job.
By default, Rails Active Job uses a memory-based thread pool to handle background jobs. This is good for development mode but not for production because, ff your Rails app is restarted you will lose all running or pending background jobs.
So, we need a backend that uses a persistent storage for Active Job.
Up until recently resque and sidekiq were the popular backends for Rails Active Job. Both resque and sidekiq use Redis as their persistent storage so you need to have a Redis cluster in your production setup in addition to the main SQL database. This is a bit of a hassle.
Solid Queue uses an SQL database like PostgreSQL or MySQL as the persistent storage so you don’t have to depend on Redis in production.
Configure Solid Queue as the Active Job backend
After installing Solid Queue, we must configure our Rails app to use Solid Queue as the Active Job backend.
Create the database migrations for Solid Queue:
$ bin/rails solid_queue:install:migrations
Update config/environments/development.rb
to use Solid Queue as the Active Job backend in the development mode.
config.active_job.queue_adapter = :solid_queue
Run the migrations:
$ bin/rails db:migrate
Run Solid Queue in development mode
Run the Rails server:
$ bin/rails server
Start Solid Queue supervisor in a separate process:
$ bundle exec rake solid_queue:start
Go to http://127.0.0.1:3000/articles/
and click on ‘New article`. Write and save the article and check your email account for an email from the Rails app.
If you have a different Rails app, check the outcome of your Active Job task.
Instead of running Solid Queue in a separate process, we can run it together with the Puma app server.
Update config/puma.rb
to run the Solid Queue with the app server:
plugin :solid_queue
Run the Rails server again:
$ bin/rails server
Now you don’t need to run Solid Queue separately.
Deploy our Rails application to production with Solid Queue
Update config/environemnts/production.rb
to use Solid Queue as the Active Job backend in production mode:
config.active_job.queue_adapter = :solid_queue
Now we can deploy this app as any other Rails app.
Check out this tutorial to deploy this Rails application on Docker using Docker compose.
Wrapping up
Running Solid Queue with Puma app server makes it easy for us to deploy this Rails application to production.
But a long-running background job can make our application feel sluggish for users.
To avoid that we must run Solid Queue as a separate process in production. If you are running this Rails app on PAAS, check out the PAAS provider’s documentation on how to run Rails background jobs in separate processes.
In part 3 of this tutorial series, we will deploy this app on Docker with Solid Queue running on a separate container.