Getting started with Judoscale on Hatchbox

Judoscale autoscales Hatchbox clusters by adding and removing servers. A Hatchbox account becomes a Judoscale team, and each cluster you link becomes a Judoscale application. Judoscale scales the cluster’s web and worker server roles independently, using request queue time for web servers and job queue latency for workers.

Sign up for a Judoscale account and select “Hatchbox” for your hosting platform.

Create a Hatchbox API token

In Hatchbox, open API Tokens, create a new token, and name it “Judoscale” (you can name it anything you want). The token belongs to your Hatchbox user, so it can access every Hatchbox account your user belongs to. The Hatchbox API also requires an active subscription or trial on the account you link.

Paste the token into Judoscale, and click “Connect to Hatchbox”.

Link an account and a cluster

After connecting, Judoscale lists the Hatchbox accounts your user belongs to. Link the account that owns the cluster you want to autoscale. That account becomes your Judoscale team.

Next, Judoscale lists the clusters in that account, with each cluster’s provider, region, and the Hatchbox apps deployed to it. Click “Link to Judoscale” next to the cluster you want to autoscale.

Linking a cluster lists its scalable roles on the team dashboard: “Web Server” for servers with the web role, and “Background Worker” for servers with the worker role. None of them autoscale yet.

How Judoscale scales Hatchbox clusters

Judoscale scales a cluster by server role, not by individual server:

• Each role needs at least one server. When Judoscale scales up, Hatchbox copies the provider, region, and size from a server that already has that role. A role with no servers can’t scale up, so keep your minimum at one or more.
• Scaling down removes the newest server. Your original servers stay put. That matters for cron: new worker servers don’t get the cron role, so your scheduled jobs keep running on the worker you started with.
• New servers take a few minutes. Hatchbox creates the server, configures it, and deploys your app before it starts serving traffic. In our testing that took about 3.5–4 minutes on clusters that deploy with Docker, and about 6.5 minutes on other clusters.
• Failed provisions are replaced. If Hatchbox fails to provision a server, Judoscale requests a replacement automatically.
• Only web and worker scale. Load balancer, database, and cron roles are left alone. A server with both the web and worker roles counts toward both.
• Counts are cluster-wide. If you deploy several Hatchbox apps to one cluster, every app runs on the servers Judoscale adds.

Installing the Judoscale package

In the Judoscale team dashboard, click the role you want to autoscale. This takes you to the Scaling page, where you’ll be prompted to complete the setup.

Choose your stack, and follow the instructions to install the package for it. The adapter package is how Judoscale collects metrics from your application, similar to a lightweight APM tool. Check the web frameworks and job backends we currently support here.

Judoscale then shows a JUDOSCALE_URL value. Add it to your app’s environment variables in Hatchbox, and deploy so every server picks it up. See Making sense of your JUDOSCALE_URL env vars.

Click “Finished and Deployed”. Judoscale will begin showing your queue metrics in the Scaling page charts.

Recommended web settings

Because a new Hatchbox server takes several minutes to arrive, the defaults we use on other platforms scale web servers down too eagerly. We recommend these settings for the Web Server role:

• Queue time: downscale below 15 ms, upscale at 20 ms.
• Utilization: downscale below 50%, upscale at 100%. Turn it on with “Configure Metrics”, and see Understanding the web utilization metric.
• Downscale delay: keep the default of 5 minutes.

Here’s why:

• Idle Hatchbox apps show a few milliseconds of queue time. With the default 5 ms downscale threshold, that noise can keep Judoscale from ever scaling down. A 15 ms downscale threshold sits above it.
• Utilization keeps busy servers around. Judoscale only scales down when every enabled metric is below its downscale threshold. Queue time can look healthy while the remaining servers are nearly saturated, and removing one would cost several minutes to recover. Utilization above 50% blocks that scale-down.

The defaults work well for the Background Worker role.

Enabling autoscaling

Scroll down the Scaling page to set your “Servers” range. There’s no “correct” range: the minimum is the capacity you always want running (at least one), and the maximum limits your costs. Under heavy load, Judoscale keeps adding servers until it reaches the maximum.

Click “Save and enable autoscaling”, and that’s it!