Refining your autoscaling behavior

Updated

Judoscale offers four settings that control autoscaling behavior.

Configuration for web autoscaling

Let’s dive into how each of these settings affect your autoscaling.

Response time range / Queue time range

The exact name of this setting is different depending on your autoscaling metrics. The default for web dynos is “response time range”. If you’ve installed a language-specific adapter library, you may have changed this to “queue time range”. For worker dynos, “queue time range” will be shown.

This setting controls when an autoscale event is triggered. Judoscale aggregates metrics every 10 seconds, and the most recent metrics are compared with this range. Judoscale’s job is to keep your metrics within this range.

If any metric exceeds your specified range, an upscale event will be triggered. Your app will scaled up if all of the following are true:

  • A recent 10-second aggregate metric has exceeded the metric range.
  • The process has not scaled up in the last three minutes.
  • The process is not scaled at or beyond the top “dyno range” setting.

For downscaling to be triggered, all recent metrics must fall below the metric range. The “downscaling delay” section will explain the downscaling logic in more detail.

Dyno range

This range specifies when you want Judoscale to stop upscaling or downscaling your application. Judoscale will autoscale your app within this range, and never beyond it.

Judoscale has no control over how your application is scaled manually (via Heroku’s GUI or CLI) or from another autoscaling service. Your dyno range only impacts Judoscale’s ability to scale your application.

If a given process is already scaled at or beyond the top of this range, Judoscale upscaling will be suppressed. If the process is scaled at or below the bottom of this range, Judoscale downscaling will be suppressed.

Dyno jumps

“Dyno jumps” controls the upscaling increment. When an upscale event is triggered, how many additional dynos do you want?

Some applications experience sudden and drastic increases in load. For these applications, adding one dyno might not be sufficient to prevent a slowdown or timeouts. Setting “dyno jumps” to 1 or 2 is sufficient for most applications, but you may find that you need even more.

Judoscale limits upscale events to one every three minutes. If you need to scale up faster, “dyno jumps” is how to do it.

Dyno jumps will not override the dyno range. For example, let’s say a process is currently running 9 dynos, with a “dyno range” of 2-10 dynos and “dyno jumps” of 2 dynos. The next upscale would take that process to 10 dynos, a “jump” of only one dyno.

Downscale delay

We don’t have a “dyno jumps” option for downscaling. Instead, “downscale delay” is for controlling how quickly your application scales down.

This settings actually controls two things:

  • How long metrics must remain below the metric range to trigger a downscale event.
  • How long to wait between subsequent downscale events.

Let’s revisit the downscaling logic. A process will be scaled down when all of the following are true:

  • All metrics in the last [downscale delay] minutes were below the metric range.
  • The process has not scaled up or down in the last [downscale delay] minutes.
  • The process is not scaled at or below the bottom “dyno range” setting.

The longer your app takes to scale down, the more dyno resources you use, so this control is all about cost savings. To optimize cost savings, decrease your downscale delay. Note that this approach could result in premature downscaling. This is generally a minor risk since Judoscale will quickly scale the process back up if needed.