Skip to content

WPB-26489 backend adminless scheduled jobs model and teardown support#5289

Open
battermann wants to merge 47 commits into
developfrom
WPB-26489-backend-adminless-scheduled-jobs-model-and-teardown-support
Open

WPB-26489 backend adminless scheduled jobs model and teardown support#5289
battermann wants to merge 47 commits into
developfrom
WPB-26489-backend-adminless-scheduled-jobs-model-and-teardown-support

Conversation

@battermann

@battermann battermann commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

Checklist

  • Add a new entry in an appropriate subdirectory of changelog.d
  • Read and follow the PR guidelines

@zebot zebot added the ok-to-test Approved for running tests in CI, overrides not-ok-to-test if both labels exist label Jun 25, 2026
@battermann battermann changed the base branch from WPB-26487-backend-scheduled-jobs-http-control-plane-using-arbiter to develop June 29, 2026 10:13
@battermann battermann force-pushed the WPB-26489-backend-adminless-scheduled-jobs-model-and-teardown-support branch from 7737422 to 4e704c1 Compare June 29, 2026 12:28
@battermann battermann marked this pull request as ready for review June 29, 2026 12:30
@battermann battermann requested review from a team as code owners June 29, 2026 12:30
@battermann battermann requested a review from Copilot June 29, 2026 12:52

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces an Arbiter-backed scheduled-jobs subsystem across services, migrating the meetings-cleanup worker to run via Arbiter and adding initial “adminless” reminder/deletion scheduled jobs, along with an internal /i/jobs UI/API mount in Brig.

Changes:

  • Add Wire.API.Jobs, Wire.JobSubsystem, and Wire.JobStore to model/schedule jobs and persist minimal app-level metadata in Postgres.
  • Replace the standalone meetings cleanup cron thread with an Arbiter worker pool (scheduledJobs.pollInterval controls polling cadence).
  • Expose Arbiter’s internal UI + REST API through Brig’s internal routes under /i/jobs and /i/jobs/api/v1.

Reviewed changes

Copilot reviewed 56 out of 57 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
services/galley/src/Galley/Env.hs Add Galley env field for passing a connection string into the job subsystem.
services/galley/src/Galley/App.hs Wire JobSubsystem + JobStore interpreters and derive a Postgres connection string for Arbiter usage.
services/galley/galley.cabal Add arbiter-core dependency for Galley.
services/galley/default.nix Add arbiter-core Nix dependency for Galley.
services/brig/src/Brig/Run.hs Adjust internal API wiring to pass env into IAPI.servantSitemap.
services/brig/src/Brig/App.hs Initialize embedded Arbiter server/UI application and store it in Brig env.
services/brig/src/Brig/API/Internal.hs Mount internal /i/jobs Raw app into Brig internal routes.
services/brig/default.nix Add Arbiter packages (arbiter-core, arbiter-servant, arbiter-servant-ui).
services/brig/brig.cabal Add Arbiter packages to Brig build-depends.
services/background-worker/test/Test/Wire/Util.hs Extend test env with arbiterConnStr placeholder.
services/background-worker/test/Test/Wire/BackendNotificationPusherSpec.hs Extend test env with arbiterConnStr placeholder.
services/background-worker/src/Wire/MeetingsCleanupWorker.hs Refactor to expose runCleanupOldMeetings instead of owning the cron thread; move metrics increment into cleanup routine.
services/background-worker/src/Wire/Effects.hs Add JobSubsystem/JobStore effects + interpreters and extend migration handling.
services/background-worker/src/Wire/BackgroundWorker/ScheduledJobs.hs New scheduled-jobs dispatcher/worker wiring for meetings cleanup + adminless jobs via Arbiter.
services/background-worker/src/Wire/BackgroundWorker/Options.hs Add scheduledJobs config section (poll interval).
services/background-worker/src/Wire/BackgroundWorker/Env.hs Add arbiterConnStr to runtime env, derived from Postgres settings.
services/background-worker/src/Wire/BackgroundWorker.hs Start the unified scheduled-jobs worker instead of the old meetings-cleanup worker thread.
services/background-worker/src/Wire/AdminlessJobsWorker.hs New adminless reminder/deletion job handlers + scheduled job cleanup in app-level store.
services/background-worker/default.nix Add arbiter-core dependency for background-worker.
services/background-worker/background-worker.integration.yaml Add integration config for scheduledJobs.pollInterval.
services/background-worker/background-worker.cabal Expose new modules and add arbiter-core dependency.
postgres-schema.sql Update schema snapshot to include scheduled jobs table and related DB objects.
nix/manual-overrides.nix Add overrides/jailbreaks for Arbiter package set.
nix/haskell-pins.nix Add Arbiter source pin and package mapping.
libs/wire-subsystems/wire-subsystems.cabal Add Arbiter deps and expose new job subsystem/store modules; add cron dep.
libs/wire-subsystems/test/unit/Wire/ConversationSubsystem/InterpreterSpec.hs Add a JobSubsystem mock interpreter for conversation subsystem unit tests.
libs/wire-subsystems/src/Wire/Options/Keys.hs Fix ASN.1 imports to use crypton-asn1-* packages explicitly.
libs/wire-subsystems/src/Wire/JobSubsystem/Workers.hs New Arbiter worker runner helpers (recurring + one-off) with explicit defaults.
libs/wire-subsystems/src/Wire/JobSubsystem/Interpreter.hs New Polysemy interpreter to schedule jobs into Arbiter and start workers.
libs/wire-subsystems/src/Wire/JobSubsystem.hs New Polysemy effect and configuration types for scheduling/starting jobs.
libs/wire-subsystems/src/Wire/JobStore/Postgres.hs New Postgres-backed interpreter for persisting minimal scheduled job metadata.
libs/wire-subsystems/src/Wire/JobStore.hs New Polysemy effect for scheduled job metadata persistence.
libs/wire-subsystems/src/Wire/ConversationSubsystem/Update.hs Schedule adminless reminder/deletion jobs when last admin leaves; add reminder event emission.
libs/wire-subsystems/src/Wire/ConversationSubsystem/Interpreter.hs Add internal actions for adminless deletion/reminder execution paths.
libs/wire-subsystems/src/Wire/ConversationSubsystem/Action.hs Factor conversation deletion into reusable removeConversation.
libs/wire-subsystems/src/Wire/ConversationSubsystem.hs Add new internal conversation actions for adminless job execution.
libs/wire-subsystems/postgres-migrations/20260624095944-extend-job-model.sql Add scheduled_jobs table + indexes migration.
libs/wire-subsystems/default.nix Add Arbiter and cron dependencies for wire-subsystems.
libs/wire-api/wire-api.cabal Expose new Jobs API modules in wire-api.
libs/wire-api/test/unit/Test/Wire/API/Conversation.hs Extend event tests for new ConvAdminlessReminder type classification.
libs/wire-api/src/Wire/API/Team/FeatureFlags.hs Accept both legacy and current JSON keys for search visibility defaults.
libs/wire-api/src/Wire/API/Routes/Internal/Jobs.hs New internal route type for mounting the embedded jobs app.
libs/wire-api/src/Wire/API/Routes/Internal/Brig.hs Add Jobs app route into Brig internal API type.
libs/wire-api/src/Wire/API/Jobs.hs New job payload types, queue names, and registry definition for Arbiter.
libs/wire-api/src/Wire/API/Event/Conversation.hs Add ConvAdminlessReminder event type and corresponding event payload schema.
libs/types-common/src/Data/Id.hs Introduce ScheduledJobId and ScheduledJob id tag.
libs/extended/src/Hasql/Pool/Extended.hs Add shared postgresqlConnectionString helper and refactor pool init to use it.
integration/test/Test/AdminlessGroups.hs Add integration coverage for adminless scheduling and Arbiter execution path.
integration/test/Notifications.hs Add helper to detect adminless reminder notifications.
flake.nix Add Arbiter git input.
flake.lock Lock Arbiter input revision/hash.
docs/src/developer/reference/config-options.md Document scheduledJobs.pollInterval.
charts/wire-server/values.yaml Add Helm value for background-worker.config.scheduledJobs.pollInterval.
charts/wire-server/templates/background-worker/configmap.yaml Render scheduledJobs config into background-worker configmap.
changelog.d/2-features/WPB-26489 Feature changelog entry for Arbiter-backed scheduled jobs + adminless jobs.
changelog.d/0-release-notes/WPB-26489 Release notes for Arbiter scheduling and /i/jobs endpoints.
cabal.project Add source-repository-package entries for Arbiter packages.

Comment thread libs/extended/src/Hasql/Pool/Extended.hs
Comment thread libs/types-common/src/Data/Id.hs
Comment thread integration/test/Test/AdminlessGroups.hs Outdated
Comment thread integration/test/Test/AdminlessGroups.hs Outdated
Comment on lines 89 to 92
_convCodeURI :: Either HttpsUrl (Map Text HttpsUrl),
_passwordHashingRateLimitEnv :: RateLimitEnv
_passwordHashingRateLimitEnv :: RateLimitEnv,
_jobsApiConnStr :: Text
}

@akshaymankar akshaymankar left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Haven't read everything, comments based on just trying to understand the arbiter stuff.

Comment thread cabal.project Outdated
Comment thread services/brig/src/Brig/App.hs Outdated
Comment thread services/brig/src/Brig/API/Internal.hs Outdated
Comment thread services/background-worker/src/Wire/Effects.hs Outdated
Comment thread services/background-worker/src/Wire/Effects.hs Outdated
Comment thread libs/wire-api/src/Wire/API/Routes/Internal/Jobs.hs Outdated
@battermann battermann force-pushed the WPB-26489-backend-adminless-scheduled-jobs-model-and-teardown-support branch from 1e6c2e7 to 7ea4fca Compare July 3, 2026 10:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ok-to-test Approved for running tests in CI, overrides not-ok-to-test if both labels exist

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants