Skip to content

feat: add revalidate logic and the new SEAL_RUN_SUMMARY notification type - #1837

Merged
davidgamez merged 6 commits into
mainfrom
seal_revalidate
Sep 15, 2026
Merged

davidgamez merged 6 commits into
mainfrom
seal_revalidate

Conversation

@davidgamez

@davidgamez davidgamez commented Sep 14, 2026

Copy link
Copy Markdown
Member

Summary:

Closes #1823. The Feed Detail page is cached for 14 days or until something calls
/api/revalidate, and only dataset events did. The seal changes nightly without any
dataset moving, so the page could show a stale seal for a fortnight.

The nightly run now compares each feed's rendered seal state — has_seal, plus each
criterion's status, grace and probation — against the pre-run seal_criterion rows, and
revalidates only the feeds that actually moved. Timestamps (evaluated_at, earned_at,
probation_ends_at, ...) are excluded: they move on every feed every night and the page
never shows them.

Revalidations are now batched (~50 feeds per request, using the endpoint's existing
feedIds array) instead of one Cloud Task per feed, so a catalogue-wide night is ~58
tasks rather than ~2900 against a 1/sec queue.

Also adds an admin.event_summary / seal_run_summary notification reporting each run's
changes to admin subscribers. No schema, API or Terraform changes.

New email notification:
Screenshot 2026-09-14 at 3 38 20 PM

Expected behavior:

A run that only advances timestamps revalidates nothing. A run where a seal or criterion
changes revalidates exactly those feeds. Admin subscribers get a seal summary the same
morning (seal runs 04:00 UTC, dispatcher 08:00 UTC).

Testing tips:

Run update_seal_of_reliability twice unchanged — the second reports feeds_changed: 0
even though evaluated_at moved. Then flip a criterion and re-run with
"revalidate": true to see the feed in changed_stable_ids.

Note: this is the first caller to send more than one id to /api/revalidate. Worth a
sanity check from @Alessandro100 that the endpoint handles a ~50-id list; if not,
WEB_REVALIDATION_CHUNK_SIZE = 1 restores per-feed behaviour without dropping anything.

Please make sure these boxes are checked before submitting your pull request - thanks!

  • Run the unit tests with ./scripts/api-tests.sh to make sure you didn't break anything
  • Add or update any needed documentation to the repo
  • Format the title like "feat: [new feature short description]". Title must follow the Conventional Commit Specification(https://www.conventionalcommits.org/en/v1.0.0/).
  • Linked all relevant issues
  • Include screenshot(s) showing how this pull request works and fixes the issue(s)

@davidgamez
davidgamez marked this pull request as ready for review September 14, 2026 19:43

@cka-y cka-y left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@davidgamez
davidgamez merged commit 5085d30 into main Sep 15, 2026
24 checks passed
@davidgamez
davidgamez deleted the seal_revalidate branch September 15, 2026 16:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Revalidate Feed Detail page on Seal status change

2 participants