Autoscaling For GoodJob Queues

Do you use GoodJob? It’s a modern gem for running background jobs in Rails—similar to Sidekiq, but it’s designed for ActiveJob and uses Postgres instead of Redis as the queue backend.

We just added GoodJob support to Judoscale, and we’re super excited! This post assumes you’re already using GoodJob in your app, and you want to autoscale your queues.

In this post we’ll cover:

• Why autoscale your GoodJob queues?
• Why Judoscale?
• How to set up autoscaling for your GoodJob queues
• Autoscale configuration options
• Monitoring your GoodJob queues
• Next steps

Why autoscale your GoodJob queues?

Two main reasons:

• You want to make sure that your background jobs are processed as quickly as possible, and you don’t want to have to manually scale your queues up and down as your job volume changes.
• You want to save money by only paying for the resources you need.

Judoscale makes it easy to achieve both of these goals.

Why Judoscale?

As I write this post, Judoscale is the only autoscaling service that supports GoodJob. If that’s not enough, here are a few more ways Judoscale is a step above other autoscalers:

• Judoscale works on all Heroku dyno types, while Heroku’s own autoscaling only works on Performance dynos.
• Judoscale will autoscale your web and worker dynos. If you have several worker processes in your app, then you can autoscale each of them independently.
• Judoscale is the fastest autoscaling service. When your queues start to back up, Judoscale will scale your dynos in seconds, not minutes.
• Judoscale is integrated into Heroku’s add-on marketplace. This makes installation a breeze, gives you integrated billing, and it means you don’t have to grant Judoscale access to your Heroku account.

How to set up autoscaling for your GoodJob queues

Okay, let’s get to it! We’ll have your queues autoscaling in minutes.

First, install the Judoscale add-on to your Heroku app:

heroku addons:create judoscale --app my-app

This will install the free “White Belt” plan, which is perfect for getting started.

Next, launch the Judoscale dashboard:

heroku addons:open judoscale --app my-app

Judoscale will walk you through the setup process. Select “GoodJob” as your job backend, and paste the provided code into your Gemfile.

Once you’ve installed the gems and deployed your app, you’ll see your GoodJob queues in the Judoscale dashboard.

If your queues are empty, it might not look like much is happening. Try enqueueing a bunch of jobs, and you’ll see your queues start to back up.

Autoscaling is turned off by default, so you’ll need to enable it. But first, let’s look at the configuration options.

Autoscale configuration options

Scroll down to the “Autoscale Trigger” section. This is where you’ll set a queue time range that will trigger autoscaling. The default range of 1–5 seconds is a good starting point. It means that your dynos will scale up when jobs are enqueued for more than 5 seconds. Your dynos will scale back down when queue time is consistently under 1 second.

You also want to select the queues you want to monitor for autoscaling. By default, Judoscale will monitor all of your queues. If you have some low-priority queues that you don’t want to trigger autoscaling, then you can uncheck them here. If you have multiple worker processes that process different queues, then this is where you’ll assign the appropriate queues to each process.

Finally, you can set a minimum and maximum number of dynos. This is useful if you want to make sure that you always have at least one dyno running, or if you want to limit the number of dynos that can be scaled up.

If your worker process is often idle, you can even scale down to zero dynos. Judoscale will automatically scale back up when jobs are enqueued. This is a great way to save money on low-volume apps like staging and demo apps.

Monitoring your GoodJob queues

Once you save your changes, autoscaling will be enabled. You can monitor your queues and scaling activity in the Judoscale dashboard.

Judoscale will automatically scale your dynos up and down as needed. The “Dyno Count” graph shows the number of dynos that are currently running. The “Queue Time” graph shows the job queue time (also called “queue latency”) for the queues you selected earlier.

Next steps

You can use the “White Belt” plan for free as long as you like. It’s limited to 20 autoscale events per month, which is perfect for experimentation or low-volume apps. If you need more, check out our pricing page to determine which plan is right for you.