Custom Worker Adapter

While Judoscale supports many job-processing frameworks across multiple languages, if you're using a job backend we don't natively support, you can create your own adapter.

Create a Custom Worker Adapter

For this example, we're going to create an adapter for a "file system" queue. Pending jobs are stored as files in tmp/queues/[queue-name].

Metrics Collector

The "metrics collector" is the core piece of adapter machinery. It is responsible for collecting the metrics (surprise!) from your job backend. It must define .adapter_config and #collect. #collect returns an array of Metric values.

Here's what our example metrics collector looks like:

# config/initializers/judoscale.rb

module Judoscale
  module FileSystemQueue
    class MetricsCollector < Judoscale::JobMetricsCollector
      def self.adapter_config
        # This comes from the add_adapter call below
        Judoscale::Config.instance.file_system_queue
      end

      def collect
        metrics = []
        t = Time.now.utc

        Dir["tmp/queues/*"].each do |queue_path|
          queue = queue_path.split("/").last

          # Queue time is based on the oldest job in the queue
          job_enqueued_times = Dir["#{queue_path}/*"].map { File.new(_1).birthtime }
          oldest = job_enqueued_times.min
          queue_time_ms = oldest ? ((t - oldest) * 1000).ceil : 0

          # This is the important part!
          metrics.push Metric.new(:qt, queue_time_ms, t, queue)
        end

        # Set log level to DEBUG to see these logs
        log_collection(metrics)

        # Also very important to return the metrics
        metrics
      end
    end
  end
end

This is a contrived example, but check out the metrics collectors for Sidekiq and Delayed Job for more realistic examples.

Adding the Adapter

Now we need to tell the core Judoscale gem about the adapter:

Judoscale.add_adapter :"judoscale-file-system-queue", {},
  metrics_collector: Judoscale::FileSystemQueue::MetricsCollector,
  # This is the name that gets exposed to `Judoscale.config`
  expose_config: Judoscale::Config::JobAdapterConfig.new(:file_system_queue)

Optional Configuration

A config block is optional, but it must come below the add_adapter call. See all config options at https://github.com/judoscale/judoscale-ruby#configuration.

Judoscale.config do |config|
  config.file_system_queue.max_queues = 50
end