Add an NGINX sidecar to expose request queue time

While the package installation alone will allow background job (worker) services to connect to Judoscale, web services require an additional step. In order to correctly determine request queue time for web services, Judoscale reads a unique header on each incoming request: X-Request-Start. Some hosting platforms automatically add this header to every incoming request. Amazon ECS does not. So we’ll need to add it ourselves!

Our recommendation is to follow the sidecar pattern, adding a second container to your web service’s task definition. This container is essentially a very minimal web-server — NGINX, for example. Its job is to simply add the header and proxy the request over to your application container as if it’d received it in the first place. We covered this more exensively in our blog post, “How Our Amazon ECS Autoscaling Works,” but the sidecar pattern is a stable and reliable pattern for header-injection.

We’ve also made our own sidecar container available from the Amazon Elastic Container Registry if you’d prefer a drop-in solution. This particular container does require that your application be mounted on port 3000. That container can be found here, but you should feel free to build your own in your own container pipeline — the Dockerfile is simply:

FROM nginx:latest

COPY nginx.conf /etc/nginx/conf.d/default.conf

and the nginx.conf is simply:

server {
     listen 80;

     location / {
         proxy_set_header X-Request-Start "t=${msec}";

         proxy_pass http://localhost:3000;
     }
 }

For our particular application, that’s all that’s required. Since we’re running on Fargate instances, all containers running within a single task share their localhost networking.

🚨 Warning

We offer our pre-fab container (and code sample above) only as an example; your specific application configuration may vary. While the sidecar pattern should work to inject headers into any application’s web service, you may have differences or preferences in port numbers, inter-container networking, and/or proxy web-server choices. As long as the X-Request-Start header is added to the request before it reaches your application, Judoscale will be able to properly determine your request queue time.

Verify Incoming Metrics

Once you’ve installed the package, setup a sidecar (for web services), and deployed your application, Judoscale will begin showing your queue metrics in the Scaling page charts.

If your web service isn’t receiving traffic, or if your worker service has no jobs waiting in queue, you won’t see any activity in the charts. Let it collect metrics while your app is under load to see queue time information.