Skip to main content

71. Bull Board for Queue Monitoring

Status: Accepted Date: 2025-07-06

Context​

Our system relies heavily on BullMQ for background job processing. With many different queues and jobs being processed by the mercury-worker, we need a simple and effective way for developers and operators to visualize the state of the queues. Trying to inspect this state by connecting directly to Redis and interpreting the raw data structures is slow, difficult, and error-prone.

Decision​

We will integrate and expose the bull-board user interface as part of the mercury-worker application. bull-board is a popular open-source web-based UI for BullMQ that provides a rich, interactive dashboard.

The bull-board UI will be served from a dedicated endpoint on the mercury-worker service (e.g., /admin/queues). It will be configured to display all the BullMQ queues that the worker interacts with, allowing users to:

  • View the number of active, waiting, completed, and failed jobs per queue.
  • Inspect the data and options of individual jobs.
  • Manually retry or delete failed jobs.
  • Pause and resume queues.

Consequences​

Positive:

  • Excellent Observability: Provides immediate, out-of-the-box visibility into the health and status of our entire job queueing system. This is invaluable for debugging and operations.
  • Easy Manual Intervention: Allows operators to quickly diagnose and resolve issues with specific jobs (e.g., retrying a job that failed due to a transient error) without needing to write custom scripts or use the CLI.
  • Fast Implementation: Integrating bull-board is fast and requires minimal configuration, providing a huge amount of value for very little development effort.

Negative:

  • Security Risk: The dashboard is a powerful administrative tool. It allows users to manipulate jobs and queues, which could be dangerous if accessed by unauthorized personnel.
  • Adds Web Server to Worker: This adds an HTTP server and a web UI to our worker process, which slightly increases its complexity and resource footprint.

Mitigation:

  • Strict Authentication: Access to the bull-board endpoint will be protected by a mandatory authentication layer. Initially, this will be simple Basic Authentication (adr://basic-auth-security), but it can be upgraded to our standard SSO/OAuth provider later. Access will be limited to authorized administrators and operators.
  • Minimal Performance Impact: The bull-board UI is lightweight and the overhead of running the web server is negligible for a process that is already I/O-heavy. The benefits to observability far outweigh this minor cost.
  • Read-Only Mode: For less privileged users, the board can be configured in a "read-only" mode, allowing them to view the state of the queues without being able to perform any actions.