Getting started with Judoscale using a Custom API Integration

Judoscale allows you to build your own custom API to integrate with any platform and autoscale your deployments. This API could be a small app you deploy on your platform of choice, a serverless worker / function deployed on Cloudflare or AWS for example, or anything in between. Sign up for a Judoscale account, and select “Custom Integration” for your hosting platform, to start creating the team.

API requirements

Authentication

An auth token is required when setting up the API, which will be sent with each request to authenticate, using the Authorization: Bearer <token> header. Judoscale will generate a token during the team setup flow that you can use, or you can provide your own.

Content-Type

The API integration expects requests and responses to use JSON format, and will include the Content-Type: application/json header when sending requests.

Status Codes

The API expects basic status codes to represent success and errors scenarios.

200 Success
401 Unauthorized
404 Not Found

Endpoints

Only two endpoints are required to be implemented on your custom API to integrate:

List Services

Returns the list of all your services, across all your applications / environments, with the following attributes:

• id: unique identifier for the service
• group: identifier to group services, like an application or environment name
• scale: the current scale for the service
• web: must be true to identify web services, can be omitted otherwise

Our backend will request this endpoint a couple of times while setting up the team, and periodically in the background afterwards, to stay in sync with your API integration and current scales.

Sample request:

GET <api_url>/services

Sample response:

[
  { "id": "web-stag", "group": "custom-app-staging", "scale": 2, "web": true },
  { "id": "worker-stag", "group": "custom-app-staging", "scale": 1 },
  { "id": "web-prod", "group": "custom-app-production", "scale": 5, "web": true },
  { "id": "worker-prod", "group": "custom-app-production", "scale": 2 }
]

Update Service Scale

Sets a new scale for the given service, with the following params:

• id: unique identifier for the service being updated
• scale: the new scale for the service

Our backend will request this endpoint whenever autoscaling is triggered, to scale your service up / down. You should implement it accordingly, updating the current scale of the service within your platform.

Sample request:

PATCH <api_url>/services/:id
{ "scale": <scale> }

A successful response (no body required) is expected.

Samples

We’ve implemented some sample APIs showcasing the required endpoints to integrate with Judoscale:

• A Ruby + Sinatra web app that manages an in-memory list of services and updates their scale via incoming API calls from Judoscale;
• A Cloudflare Worker sample that communicates with the Heroku Platform API to autoscale an application, which could be integrated with any platform that provides an API.

https://github.com/judoscale/judoscale-custom-api-sample

Setting Up a Team with your API

Once you have a basic working API that Judoscale can talk to, we can set it up as a new team, by providing the following pieces of information:

• A name to identify your new team
• The base URL to communicate with the API
• The token to authenticate each request with the API (one will be generated for you, but you can set your own)

Once you’re done, click “Connect to your API”, and we’ll attempt to reach the API with the provided information. If there’s any trouble we’ll report back the error, otherwise your team will be created and you’ll be presented with the list of groups/apps we identified from the services returned via the API.

Select the first app you want to link to Judoscale. This will add the selected app as a Judoscale application, automatically linking all the services that belong to it, and take you to the team dashboard.

Here you’ll see all of your linked apps and their respective services, the number of running instances, and their autoscaling status within Judoscale. If this is your first time linking an app, none will be autoscaling yet.

Installing the Judoscale package

In the Judoscale team dashboard, click the service you want to autoscale. This takes you to the Scaling page, where you’ll be prompted to complete the setup for that service.

Choose your stack information, and follow the instructions to install the package specific to your stack. The adapter package is how Judoscale collects metrics from your application, similar to a lightweight APM tool. Check the web frameworks and job backends we currently support here.

You’ll also need to set a JUDOSCALE_URL environment variable in your application settings. This is how the Judoscale package knows where to send metrics and link those metrics to the correct service.

Once you’ve installed the package and deployed your application, click “Finished and Deployed” to complete the setup. Judoscale will begin showing your queue metrics in the Scaling page charts.

If your web service isn’t receiving traffic, or if your worker service has no jobs waiting in queue, you won’t see any activity in the charts. Let it collect metrics while your app is under load to see queue time information.

Configuring and enabling autoscaling

Now that Judoscale is monitoring your service, you’re ready to autoscale!

Scroll down the Scaling page to review your autoscale settings. The default settings are usually a good starting point for most services, but you’ll want to customize your “Instances” range based on how high and low you’re comfortable scaling.

There’s no “correct” range here. Judoscale defaults to the initial scale as the minimum, to avoid downscaling as soon as autoscaling is enabled. Some teams are okay with having a single instance running under light load, while others always want multiple instances running, and they’ll set the minimum to two or higher. The maximum is really about limiting costs. Remember that under heavy load (high queue times), Judoscale will continue scaling your service up until the max instances is reached.

Scroll down and click “Save and enable autoscaling”, and that’s it!