Merge remote-tracking branch 'origin/main' into claude/issue-5189-publish-gate-backstop

Author: claude

.changeset/email-outbox-boot-sweep.md

@@ -0,0 +1,50 @@
+---
+"@objectstack/plugin-email": minor
+---
+
+fix(plugin-email): `sys_email` rows stranded at `queued` are swept at boot, and a failed drain says so at `error` (#5161)
+
+`status: 'queued'` had exactly one consumer: the `afterInsert` outbox drain that
+fires during the insert itself (plus, since #5160, the `email.send.async` job
+`send()` publishes). Nothing ever looked at such a row again. A process that
+died between the insert and the delivery — or a drain whose delivery threw —
+left the row at `queued` **forever**: a state named after a queue that had no
+reader, while the caller had already been told the message was accepted.
+
+**A once-per-boot sweep is now that reader.** At `kernel:ready`, after the
+registries are settled and the `email.send.async` subscriber is attached,
+`sweepStrandedOutbox` picks up `sys_email` rows still at `queued` and advances
+them:
+
+- **durable queue delivery on** → the row is published as an `{ rowId }` job to
+  `email.send.async` through the same producer, options and
+  `sys_email:<id>` idempotency key `send()` uses, so a row that still has a
+  pending job collapses onto it instead of putting a second worker on it;
+- **inline delivery** → the row is delivered and finalized in place (`sent` /
+  `failed`), which is what the drain hook would have done had the process lived.
+
+Only rows **older than five minutes** are eligible. A row inserted seconds ago
+is not stranded, it is someone's in-flight work — this process's `send()`, its
+deferred drain hook, or the same on another instance — and sweeping it would
+send that message twice. (Age, not "created before this boot": one instance's
+boot time says nothing about a sibling's row inserted a second ago.) Rows this
+process is delivering right now, and rows that already carry a `message_id`, are
+skipped. The batch is bounded at 500 rows per boot, oldest first, and says so
+when it truncates. One `info` line reports the counts; boot does **not** wait on
+the sweep, and a sweep that cannot run reports at `error` rather than relying on
+`kernel:ready` error propagation.
+
+**Drain-hook failures are now `error`, not `warn`.** A drain that throws means
+the mail was not sent while the insert reported success and the row still reads
+`queued` — the durability class the degradation-log-level rule pins at `error`.
+Both lines now name the consequence (this message was NOT sent, the row stays at
+`queued`) and the fix (the boot sweep picks it up on the next restart; turn on
+durable queue delivery to have failures retried and dead-lettered instead).
+`deliverPersistedRow` joins `DURABILITY_CRITICAL_CALLEES`, so a future `catch`
+that quietly downgrades it fails `pnpm check:durability-log-level`.
+
+New exports: `sweepStrandedOutbox`, `OUTBOX_OBJECT`, `OUTBOX_SWEEP_MIN_AGE_MS`,
+`OUTBOX_SWEEP_LIMIT`, `EmailService.enqueuePersistedRow`, and
+`EmailServicePlugin.outboxSweepSettled` (the sweep's promise, for callers that
+need determinism). The normal `send()` → deliver path is byte-for-byte
+unchanged.

.changeset/job-queue-completed-retention.md

@@ -0,0 +1,52 @@
+---
+"@objectstack/platform-objects": minor
+"@objectstack/service-queue": minor
+---
+
+fix(service-queue): `sys_job_queue` no longer grows forever — `completed` rows expire on a declared 7-day retention (#5179)
+
+`DbQueueAdapter` marked a delivered message `status: 'completed'` and then
+**nothing ever touched that row again**. `purge()` had zero production callers
+(tests only), `purgeFailed()` is a manual dead-letter API, and the object
+declared no lifecycle policy at all — so every queue delivery left a permanent
+row, which since #5160 means one permanent row per queued email.
+
+`sys_job_queue` now declares an ADR-0057 policy and the platform
+`LifecycleService` enforces it on its existing hourly sweep:
+
+```ts
+lifecycle: {
+  class: 'transient',
+  retention: { maxAge: '7d', onlyWhen: { status: 'completed' } },
+}
+```
+
+**Only `completed` rows are swept.** `pending` / `running` are live work, and
+`failed` / `dlq` are the dead-letter queue — they exist to wait for a human, so
+they are never deleted automatically at any age. `listFailed()` / `replay()` /
+`purgeFailed()` remain the only way a dead letter leaves the table. This is
+also why the policy is `retention` (age + row filter) rather than a `ttl` on
+`completed_at`: TTL has no row filter, and `dlq` rows stamp `completed_at` too.
+
+**No new configuration, and no new sweeper.** ADR-0057 §3.3 puts one reaper in
+the platform rather than one per plugin — the same call the sibling
+`sys_job_run` (30d) already makes. Any kernel with a data engine already runs
+it, its per-sweep `[lifecycle] sweep: … ~N rows reaped` line now accounts for
+this table too, and the window is overridable per environment through the
+`lifecycle` settings namespace without touching code.
+
+**The dedup window is now an enforced invariant, not a coincidence.** Publish
+dedups against a terminal row by comparing its `created_at` to
+`idempotencyWindowMs` (default 24h), and the reaper cuts off on that same
+`created_at` axis — so retention (7d) ≥ dedup window is what keeps "duplicate
+publishes inside the window are suppressed" true. `DbQueueAdapter` reads the
+declared window (new export `completedRetentionWindowMs()`) and **throws at
+construction** if `idempotencyWindowMs` is configured longer than it, instead of
+silently degrading into duplicate deliveries days later. If you raise
+`idempotencyWindowMs` past 7 days, raise the object's declared retention (or the
+`lifecycle` settings override) to match — the error message names both numbers.
+
+`class: 'transient'` is deliberate: `telemetry`/`event`/`audit` classes
+relocate their table to the dedicated `telemetry` datasource wherever one is
+registered (ADR-0057 §3.6), and moving a live work queue's storage would be a
+migration, not a cleanup.

.changeset/record-delete-share-cascade.md

@@ -0,0 +1,57 @@
+---
+"@objectstack/plugin-sharing": patch
+---
+
+fix(sharing): deleting a record now revokes every `sys_record_share` row on it, whatever the source (#5103)
+
+A share row says "principal P has level L on (object O, record R)". Delete R and
+the row describes nothing — yet until now it stayed in the table forever.
+
+#4779 (PR #5102) bound an `afterDelete` for this, but inside the sharing-RULE
+package, where two conditions fenced it in: it revokes only `source: 'rule'`
+rows, and it binds only on objects that appear in `sys_sharing_rule`. So an
+object that uses nothing but MANUAL shares — a `sharingModel: 'private'` object
+with no rule ever configured — had no delete hook at all, and **manual share +
+record delete = a permanent orphan**.
+
+Today the harm is bounded, and only because record ids are never reused: the
+`record_id IN (…)` predicate `buildReadFilter` emits matches nothing. Nothing
+enforces that assumption. A custom primary key, an import that preserves ids, or
+any future id recycling turns every one of those rows into a real privilege
+escalation — a new record landing on a recycled id inherits the dead record's
+recipients outright. Secondarily, `sys_record_share` grew without bound and
+Setup's Record Shares list showed rows pointing at nothing.
+
+**What changed**
+
+- **A record-delete cascade on every sharing-capable object.** `plugin-sharing`
+  binds one `beforeDelete`/`afterDelete` pair with no object filter and judges
+  the object's sharing posture from its `sharingModel` metadata *per delete*.
+  Nothing is enumerated at boot, so nothing goes stale: an object that gains
+  `sharingModel` at runtime is covered on its very next delete, with no rebind.
+  Bounded deletes (a scalar id, an `$in` list, or a predicate matching at most
+  1000 rows) are revoked synchronously and set-based; an unbounded one queues an
+  object-scoped orphan sweep instead. System-context deletes cascade too.
+- **A boot-time orphan sweep keyed on record existence.** On
+  `kernel:bootstrapped`, share rows whose RECORD no longer exists are revoked —
+  historical orphans, rows a failed hook missed, and the one posture the cascade
+  deliberately skips (an unmarked system object). This is a different question
+  from the existing `sweepOrphanedRuleGrants`, which asks whether the RULE row
+  still exists and therefore can never see a manual share. Bounded per boot:
+  keyset pages, one batched existence probe per object per page, and a scan cap
+  that reports when it stopped early. An object whose existence probe FAILS has
+  its rows left in place — "could not ask" is never read as "the record is gone".
+
+**What did not change**
+
+Rule *recompute* still never touches a manual share. That boundary (#5102) is
+the point: while the record exists, a manual grant is a human decision no rule
+evaluation may overrule. Only the record's DELETION revokes it, and only because
+there is no longer anything to have access to.
+
+New exports for hosts that compose the plugin by hand:
+`bindRecordShareCascade` / `unbindRecordShareCascade`,
+`objectCanCarryRecordShares`, `SharingService.revokeSharesForDeletedRecords`,
+`SharingService.sweepOrphanedRecordShares`, and `effectiveSharingModel`. Nothing
+was removed or renamed; the standard `SharingServicePlugin` composition needs no
+changes.

packages/platform-objects/src/audit/sys-job-queue.object.ts

@@ -22,6 +22,9 @@ import { ObjectSchema, Field } from '@objectstack/spec/data';
  * Writers: `DbQueueAdapter` (publish/lease/complete/fail).
  * Readers: Studio DLQ view, ops dashboards, the adapter's worker loop.
  *
+ * Retention: `completed` rows are swept by the platform LifecycleService —
+ * see the `lifecycle` block below (#5179).
+ *
  * @namespace sys
  */
 export const SysJobQueue = ObjectSchema.create({
@@ -31,6 +34,56 @@ export const SysJobQueue = ObjectSchema.create({
   icon: 'inbox',
   isSystem: true,
   managedBy: 'engine-owned',
+
+  /**
+   * [ADR-0057 §3.1/§3.3, #5179] The queue table only ever GREW: the adapter
+   * marks a delivered message `completed` and nothing ever touched the row
+   * again (`purge()` had zero production callers, `purgeFailed()` is a manual
+   * dead-letter API). Since #5160 that is one permanent row per email.
+   *
+   * Bounded declaratively rather than by a sweeper inside `DbQueueAdapter`:
+   * ADR-0057 §3.3 puts ONE reaper in the platform (`LifecycleService`), not N
+   * per-plugin ones — the same call the sibling `sys_job_run` already makes.
+   * That the writer is the adapter itself (never user data) is what makes an
+   * unattended delete safe here; the declaration is where an operator can see
+   * the window, and `lifecycle` settings can override it per environment
+   * without a code change.
+   *
+   * `onlyWhen: { status: 'completed' }` is the whole safety story:
+   *   - `pending` / `running` are LIVE work — reaping them would drop
+   *     undelivered messages;
+   *   - `dlq` / `failed` are the dead-letter surface and exist precisely to
+   *     wait for a human (`listFailed` / `replay` / `purgeFailed`), so they
+   *     are never swept automatically, at any age.
+   * This is also why the policy is `retention` (age by `created_at` + row
+   * filter) and not `ttl` on `completed_at`: TTL has no row filter, and `dlq`
+   * rows stamp `completed_at` too — a TTL would eat the dead-letter queue.
+   *
+   * Window = 7d, and it MUST stay ≥ the adapter's idempotency window
+   * (`DbQueueAdapterOptions.idempotencyWindowMs`, default 24h): publish
+   * dedups against terminal rows by comparing `created_at` to that window
+   * (`db-queue-adapter.ts`), and the Reaper cuts off on the very same
+   * `created_at` axis — so a retention ≥ the dedup window means a row the
+   * dedup check still needs can never have been reaped, with no clock skew
+   * between the two rules. 7d gives a week of delivery history for debugging
+   * and 7× headroom over the default dedup window. `DbQueueAdapter` reads
+   * this declaration and refuses to start when the two are configured the
+   * wrong way round, so the invariant cannot drift apart silently.
+   *
+   * `class: 'transient'` ("workflow / ephemeral state" — ADR-0057 §3.1), not
+   * `telemetry`: this is live work state, not a log, and per §3.6 a
+   * `telemetry`/`event`/`audit` class RELOCATES the table to the dedicated
+   * `telemetry` datasource wherever one is registered. Moving a live queue's
+   * store is a migration, not a cleanup — `transient` deliberately stays on
+   * the primary.
+   */
+  lifecycle: {
+    class: 'transient',
+    retention: {
+      maxAge: '7d',
+      onlyWhen: { status: 'completed' },
+    },
+  },
   description: 'Durable job/message queue including dead letters',
   displayNameField: 'queue',
   nameField: 'queue', // [ADR-0079] canonical primary-title pointer (mirrors deprecated displayNameField)