Add Redis observability metadata for a simple UI (dead set + counters)

[darkamenosa]

Hi Samuel,

So my goal is simple, I'm trying to build a UI similar Sidekiq or Mission Control Jobs (Solid Queue) for async-job-adapter-active_job-server using Redis. When using UI, it is just simpler and faster to trace error on production or even development environment.

I read this codebase, and I found that metadata stored on Redis is not enough, so I learn from Sidekiq. Not sure, it is intended or not. I just try to make a pull request to see if I can help.

Summary
This PR adds optional, bounded metadata in Redis so we can build a lightweight UI similar to Sidekiq’s “Dead” jobs and basic stats — without adding a
database. The goal is only to store enough info in Redis to power a small dashboard (recent failures + processed/failed counters), leaving the actual UI
for a follow-up.

What’s included

• New Redis keys (under the existing prefix, default async-job)
  • :dead (ZSET): recent failed jobs, newest first (score = failed_at).
  • :stat:processed (STRING): count of successfully processed jobs.
  • :stat:failed (STRING): count of failed executions.
• Server options (all optional; sensible defaults)
  • stats: enable counters (default: true)
  • dead_enabled: enable failure “morgue” (default: true)
  • dead_max: max number of dead entries to keep (default: 1000)
  • dead_timeout: optional time-based trim in seconds (default: nil)
  • failure_backtrace_limit: backtrace lines to retain (default: 10)
• Behavior
  • On success: increments :stat:processed.
  • On failure: writes a compact JSON entry to :dead, trims to dead_max and (optionally) by age, increments :stat:failed, then retries
    as before.
• Docs and tests
  • Guides updated with schema, examples, and configuration.
  • Tests cover recording failures, trimming behavior, and counter increments.

Dead entry format (compact JSON)

• jid, queue, class, args
• error_class, error_message (truncated), error_backtrace (limited)
• failed_at (float timestamp)

Example Redis queries (for a simple UI)

• Latest failures: ZREVRANGE :dead 0 19 WITHSCORES
• Stats: GET :stat:processed and GET :stat:failed
• Active processing lists: SCAN 0 MATCH :processing:* COUNT 100

Why

• I want to build a small Redis-only UI (Sidekiq-style) to browse recent failures and see basic throughput counters. This keeps storage minimal, bounded,
  and easy to query.

Compatibility

• No breaking changes. All features are opt-in via flags (enabled by default, can be turned off).
• Storage is bounded (by size and optionally by time).

Limitations / follow-ups

• This PR only emits metadata; it doesn’t ship a UI.
• Future PRs could add a tiny web view or CLI to browse :dead, show counts, and list active workers/queues.

How to test locally

• Run tests: bundle exec bake test
• Manual poke:
  • Trigger a failing job, then:
    • ZREVRANGE :dead 0 0 should return the latest failure JSON.
    • GET :stat:processed / GET :stat:failed should be >= 0.

Checklist

• Docs updated
• Tests cover new behavior
• Defaults keep storage bounded
• Flags allow disabling if not needed

Happy to tweak names, defaults, or JSON shape if you have preferences.