feat(registry): index, signed sync, catalog API, and consolidation refactors

.rubocop.yml

@@ -5,6 +5,7 @@ plugins:
   - rubocop-thread_safety
 
 AllCops:
+  TargetRubyVersion: 4.0
   DisplayCopNames: true
   NewCops: enable
   Exclude:
@@ -38,6 +39,10 @@ Style/Documentation:
   AllowedConstants:
     - App
 
+Style/ItBlockParameter:
+  Enabled: true
+  EnforcedStyle: allow_single_line
+
 RSpec/SpecFilePathFormat:
   Exclude:
     - 'spec/html2rss/web/app/*_spec.rb'

AGENTS.md

@@ -54,21 +54,63 @@ See [docs/design-system.md](docs/design-system.md) for visual rules.
 - **No host execution:** All commands MUST run inside the Dev Container via `make` or `bundle exec`.
 - **No skipped quality gate:** Opening a PR without a green Dev Container gate is forbidden. If the gate cannot run, do not open the PR; fix the environment or hand off with explicit blocker + next command for the user.
 
+## Ruby 4 Style
+
+**Ruby 4.0+ only** (see `.tool-versions`). No Ruby 3.x backward-compat shims, guards, or dual-path APIs.
+
+### Baseline
+
+- `# frozen_string_literal: true` on every `.rb` file
+- Plain Ruby — no ActiveSupport
+- Keyword arguments for public multi-arg APIs
+- Typed YARD on public methods in `app/` (`@param`, `@return`) — enforced by `make yard-verify-public-docs`
+
+### Modern syntax (prefer consistently)
+
+| Idiom | Use instead of |
+| --- | --- |
+| Leading `&&` / `\|\|` at line start (Ruby 4) | Trailing operators on long wrapped conditions |
+| `it` in single-parameter blocks | `{ \|x\| x.foo }` when block has one arg only |
+| Pattern matching (`in`, `case … in`) | Deep `if/elsif` chains on shape |
+| `Data.define` | OpenStruct / hand-rolled structs |
+| `filter_map`, `index_by`, `then`, `match?` | Verbose `map`/`compact`, nested `if`, `=~` |
+| Endless `def` | One-line pure helpers when RuboCop allows |
+| Core `Set` (no `require 'set'`) | Array membership/diff on growing collections |
+
+### Performance (agent defaults)
+
+- **Set** for catalog/diff/membership when sizes can grow
+- **Memoize** repeated `ENV.fetch` / pure computations on hot paths
+- **One owner** for duplicated helpers — dedupe before splitting into new files
+- **Functional iterators** over imperative loops
+- **No metric-driven micro-methods** whose only purpose is satisfying RuboCop metrics
+- Do **not** document ZJIT/Ruby Box/Ractor as defaults
+
+### Web-specific deltas
+
+- Prefer `class << self` + `private` over `module_function` (see docs/README Architectural Constraints)
+- Do not use `send(...)` to reach private APIs in app code or specs
+- Specs: table-drive matrices; `:aggregate_failures` for discriminating multi-assert examples
+- LOC: dedupe/unify before extracting — new files only when they buy a real seam or test surface
+
 ## Config catalog API
 
-Public feed-directory metadata for embedded and local configs.
+Public feed-directory metadata from verified registry bundles and local `feeds.yml` entries.
 
 | Item | Detail |
 | --- | --- |
 | Endpoint | `GET /api/v1/configs` |
 | Flag | `CONFIG_CATALOG_ENABLED` (default `true`; set `false` to disable) |
 | Disabled response | `404` with `{ "error": "catalog_disabled" }` |
-| Embedded entries | `Html2rss::Configs::Catalog.entries` — do not re-walk YAML in the handler |
-| Local entries | `Catalog::Merge` includes `feeds.yml` feeds only when `directory.title` is set |
+| Registry entries | `Registry::Index.current.catalog_rows` — loads signed bundles from `config/registries.yml`; adds `source: registry`, `registry: <id>` |
+| Local entries | `Registry::Index` catalog rows include `feeds.yml` feeds only when `directory.title` is set (`source: local`) |
+| Per-registry privacy | `catalog: false` in `registries.yml` omits that registry from the API (feeds still served) |
 | Starter feeds (UI) | Frontend `selectStarterFeeds` when feed creation is disabled; catalog find uses full catalog when enabled |
 | Catalog find | `findCatalogEntries` → multi-hit list under create URL; links via `catalogFeedHref` (path + defaults) |
 | CORS | Route-scoped on `/api/v1/configs` only (`GET`, `OPTIONS`) |
-| Root metadata | `GET /api/v1/` exposes `instance.catalog: { enabled, url }` |
+| Root metadata | `GET /api/v1/` exposes `instance.catalog: { enabled, url }` and `instance.registries` sync status |
 | Contract SSOT | Request specs under `spec/html2rss/web/api/v1_spec.rb` and generated `public/openapi.yaml` |
 
+Registry sync: `bin/registry-sync --status`; boot seed + optional sync via `Registry::Sync.boot!`. See [docs/README.md](docs/README.md#registry-sync-runbook).
+
 After handler or envelope changes: `make openapi` and `make ci-ready`.

CONTEXT.md

@@ -54,3 +54,12 @@ Audit channel: snake_case `security_event` with IP / user-agent / token hash. Au
 
 ### LogEvent
 Shared emit plumbing for both channels (`RequestContext`, `LogSanitizer`, `AppLogger` / Sentry). Not a third public facade.
+
+### Registry Index
+Backend merge owner for registry bundles and local `feeds.yml` feeds. Builds catalog wire rows (`Registry::Index::CatalogRow`), enforces load-time trust and channel-domain allowlists, and serves `config_for` / `catalog_rows` / `status`.
+
+### Registry Sync
+Backend orchestration for fetch → verify → stage/promote of signed registry bundles. Owns boot seeding, background refresh, CLI exit codes, and catalog-change telemetry after promotion.
+
+### Sync Transport
+Backend HTTPS fetch, sync URL resolution (GitHub releases and channel defaults), and manifest version gating used by `Registry::Sync` and parse-time config resolution.

Dockerfile

@@ -78,6 +78,7 @@ RUN apk add --no-cache \
   && mkdir -p /app \
   && mkdir -p /app/tmp/rack-cache-body \
   && mkdir -p /app/tmp/rack-cache-meta \
+  && mkdir -p /app/data/registries \
   && chown "$USER":"$USER" -R /app
 
 WORKDIR /app
@@ -89,6 +90,7 @@ COPY --chown=$USER:$USER bin/docker-healthcheck ./bin/docker-healthcheck
 COPY --chown=$USER:$USER Gemfile Gemfile.lock app.rb config.ru ./
 COPY --chown=$USER:$USER app ./app
 COPY --chown=$USER:$USER config ./config
+COPY --chown=$USER:$USER app/registries/seed ./app/registries/seed
 COPY --chown=$USER:$USER public ./public
 COPY --from=frontend-builder --chown=$USER:$USER /app/frontend/dist ./frontend/dist
 

Gemfile

@@ -4,13 +4,8 @@ source 'https://rubygems.org'
 
 git_source(:github) { |repo_name| "https://github.com/#{repo_name}" }
 
-gem 'html2rss', '~> 0.27'
-# gem 'html2rss', github: 'html2rss/html2rss', branch: 'master'
-gem 'html2rss-configs', github: 'html2rss/html2rss-configs'
-
-# Use these instead of the two above (uncomment them) when developing locally:
-# gem 'html2rss', path: '../html2rss'
-# gem 'html2rss-configs', path: '../html2rss-configs'
+# Until rubygems 0.28.0: git branch; local monorepo: BUNDLE_LOCAL__HTML2RSS=/path/to/html2rss
+gem 'html2rss', github: 'html2rss/html2rss', branch: 'feat/registry-v1'
 
 gem 'base64'
 gem 'rack-cache'

Gemfile.lock

@@ -1,9 +1,29 @@
 GIT
-  remote: https://github.com/html2rss/html2rss-configs
-  revision: 89f3604ac4c8ecadefd8ccda4da60dff42648cd5
+  remote: https://github.com/html2rss/html2rss
+  revision: b32ec095bd366075297f9b7f59cf55db6b665d2a
+  branch: feat/registry-v1
   specs:
-    html2rss-configs (0.2.0)
-      html2rss
+    html2rss (0.27.1)
+      addressable (~> 2.7)
+      brotli
+      dry-validation
+      faraday (> 2.0.1, < 3.0)
+      faraday-follow_redirects
+      faraday-gzip (~> 3)
+      kramdown
+      mcp (~> 1.2)
+      mime-types (> 3.0)
+      nokogiri (>= 1.10, < 2.0)
+      rack (~> 3.0)
+      rackup (~> 2.0)
+      regexp_parser
+      reverse_markdown (~> 3.0)
+      rss
+      sanitize
+      thor
+      tzinfo
+      webrick (~> 1.9)
+      zeitwerk
 
 GEM
   remote: https://rubygems.org/
@@ -103,27 +123,6 @@ GEM
       net-http (~> 0.5)
     hana (1.3.7)
     hashdiff (1.2.1)
-    html2rss (0.27.1)
-      addressable (~> 2.7)
-      brotli
-      dry-validation
-      faraday (> 2.0.1, < 3.0)
-      faraday-follow_redirects
-      faraday-gzip (~> 3)
-      kramdown
-      mcp (~> 1.2)
-      mime-types (> 3.0)
-      nokogiri (>= 1.10, < 2.0)
-      rack (~> 3.0)
-      rackup (~> 2.0)
-      regexp_parser
-      reverse_markdown (~> 3.0)
-      rss
-      sanitize
-      thor
-      tzinfo
-      webrick (~> 1.9)
-      zeitwerk
     i18n (1.15.2)
       concurrent-ruby (~> 1.0)
     io-console (0.9.2)
@@ -146,7 +145,7 @@ GEM
     loofah (2.25.2)
       crass (~> 1.0.2)
       nokogiri (>= 1.12.0)
-    mcp (1.2.0)
+    mcp (1.3.0)
       json_schemer (>= 2.4)
     mime-types (3.7.0)
       logger
@@ -319,8 +318,7 @@ PLATFORMS
 DEPENDENCIES
   base64
   climate_control
-  html2rss (~> 0.27)
-  html2rss-configs!
+  html2rss!
   irb
   puma
   rack-cache
@@ -377,8 +375,7 @@ CHECKSUMS
   faraday-net_http (3.4.4) sha256=0e78af151747ed1b00f33e25973b4bc220d7f16c00c39676817c8b12331eb588
   hana (1.3.7) sha256=5425db42d651fea08859811c29d20446f16af196308162894db208cac5ce9b0d
   hashdiff (1.2.1) sha256=9c079dbc513dfc8833ab59c0c2d8f230fa28499cc5efb4b8dd276cf931457cd1
-  html2rss (0.27.1) sha256=a13f9c0d47f4c40038fc3f4b4dd452051385c9e1bc84c921024b730b3f42cd41
-  html2rss-configs (0.2.0)
+  html2rss (0.27.1)
   i18n (1.15.2) sha256=00f9eb62412fe593b2a65a97daa75300d37abb8f7202ec748e94b6d46a9dd1b5
   io-console (0.9.2) sha256=efa74f891dd03c0939a931dfc6e74c2813d904763d456ea9762b0525e748db08
   irb (1.18.0) sha256=de9454a0703a54704b9811a5ef31a60c86949fbf4013fcf244fabc7c775248e3
@@ -389,7 +386,7 @@ CHECKSUMS
   lint_roller (1.1.0) sha256=2c0c845b632a7d172cb849cc90c1bce937a28c5c8ccccb50dfd46a485003cc87
   logger (1.7.0) sha256=196edec7cc44b66cfb40f9755ce11b392f21f7967696af15d274dde7edff0203
   loofah (2.25.2) sha256=2007f746959ac65552456e04b433e83deb22759ab38c838b4445c70e43425918
-  mcp (1.2.0) sha256=af75a270fbcbff5db74992e1d0664cfe7e2aa7f89fa9b31592bba40e9f554ba6
+  mcp (1.3.0) sha256=9395aa3a054eb8986b7714ec8abb25b533729af896f4875da4c4cb7a3024fbae
   mime-types (3.7.0) sha256=dcebf61c246f08e15a4de34e386ebe8233791e868564a470c3fe77c00eed5e56
   mime-types-data (3.2026.0701) sha256=cd8811e1fb89d836499ba0582368a10ee74cef929ba956d1d5ddca045e6a730f
   minitest (6.0.6) sha256=153ea36d1d987a62942382b61075745042a2b3123b1cd48f4c3675af9cc7d6f1

app/registries/seed/official/configs/phys.org/weekly.yml

@@ -0,0 +1,25 @@
+registry:
+  id: phys.org/weekly
+directory:
+  topics:
+    - science
+  title: "Phys.org — Weekly"
+  summary: "Top science news of the week from Phys.org."
+channel:
+  language: en
+  title: "Phys.org — Weekly"
+  url: https://phys.org/weekly-news/
+  time_zone: Europe/London
+  ttl: 1440
+selectors:
+  items:
+    selector: ".sorted-news-list .sorted-article-content"
+  title:
+    selector: "h4"
+  category:
+    selector: ".text-info"
+  categories:
+    - category
+  url:
+    selector: ".news-link"
+    extractor: "href"

app/registries/seed/official/configs/support.apple.com/en_gb_ht201222.yml

@@ -0,0 +1,34 @@
+registry:
+  id: support.apple.com/en_gb_ht201222
+directory:
+  topics:
+    - tech
+    - security
+  title: "Apple Support — Security releases"
+  summary: "Apple security update and release notes (HT201222 / related)."
+strategy: botasaurus
+channel:
+  title: "Apple Support — Security releases"
+  url: https://support.apple.com/en-gb/100100
+  language: en
+  ttl: 360
+  time_zone: UTC
+request:
+  botasaurus:
+    wait_for_selector: ".table-wrapper table tbody tr a"
+    wait_timeout_seconds: 20
+selectors:
+  items:
+    selector: ".table-wrapper table tbody > tr:not(:first-child)"
+    enhance: false
+  title:
+    selector: a
+  url:
+    selector: a
+    extractor: href
+  description:
+    selector: "td:nth-child(2)"
+  published_at:
+    selector: "td:nth-child(3)"
+    post_process:
+      - name: parse_time

app/registries/seed/official/manifest.json

@@ -0,0 +1,10 @@
+{
+  "format": "registry.v1",
+  "registry_id": "official",
+  "version": "test-fixture",
+  "public_key_id": "test",
+  "files": {
+    "configs/phys.org/weekly.yml": "0e05fa9a95ec56bef4b4363b2f044ab69642fcda9acd91ebf8d28b39141a963d",
+    "configs/support.apple.com/en_gb_ht201222.yml": "53e38a6b7d088e0c79b19a7dd6db9b009c6b5c0b6690f504c7160b0c32b71e41"
+  }
+}

app/web/api/v1/configs.rb

@@ -17,7 +17,7 @@ def index(_router)
               entries, duration_ms = build_entries
               emit_success(entries.size, duration_ms)
               success_payload(entries)
-            rescue Html2rss::Configs::Catalog::MissingDirectoryTitle => error
+            rescue Html2rss::Registry::CatalogBuilder::MissingDirectoryTitle => error
               emit_failure(error)
               raise
             end
@@ -26,7 +26,7 @@ def index(_router)
 
             def build_entries
               started = Process.clock_gettime(Process::CLOCK_MONOTONIC)
-              entries = Html2rss::Web::Catalog::Merge.call
+              entries = Registry::Index.current.catalog_rows
               duration_ms = ((Process.clock_gettime(Process::CLOCK_MONOTONIC) - started) * 1000).round
               [entries, duration_ms]
             end

app/web/api/v1/root_metadata.rb

@@ -27,16 +27,40 @@ def build(router)
             # @return [Hash{Symbol=>Object}]
             def instance_payload(router)
               {
-                feed_creation: {
-                  enabled: Flags.auto_source_enabled?,
-                  access_token_required: Flags.auto_source_enabled?
-                },
-                catalog: {
-                  enabled: Flags.config_catalog_enabled?,
-                  url: "#{router.base_url}/api/v1/configs"
-                }
+                feed_creation: feed_creation_payload,
+                catalog: catalog_payload(router),
+                registries: registry_status_rows
+              }
+            end
+
+            # @return [Hash{Symbol => Object}]
+            def feed_creation_payload
+              {
+                enabled: Flags.auto_source_enabled?,
+                access_token_required: Flags.auto_source_enabled?
+              }
+            end
+
+            # @param router [Roda::RodaRequest]
+            # @return [Hash{Symbol => Object}]
+            def catalog_payload(router)
+              {
+                enabled: Flags.config_catalog_enabled?,
+                url: "#{router.base_url}/api/v1/configs"
               }
             end
+
+            # @return [Array<Hash{Symbol => Object}>]
+            def registry_status_rows
+              Registry::Index.current.status.map do |entry|
+                {
+                  id: entry.id,
+                  version: entry.version,
+                  updated_at: entry.updated_at&.utc&.iso8601,
+                  sync_mode: entry.sync_mode.to_s
+                }
+              end
+            end
           end
         end
       end

app/web/boot/setup.rb

@@ -24,6 +24,7 @@ def call!
             configure_request_service!
             configure_runtime_logging!
             configure_gem_defaults!
+            configure_registry!
             log_startup!
           end
 
@@ -76,6 +77,11 @@ def configure_runtime_logging!
             Rack::Timeout::Logger.logger = AppLogger.logger
           end
 
+          # @return [void]
+          def configure_registry!
+            Registry::Sync.boot!
+          end
+
           # @return [void]
           def log_startup!
             AppLogger.logger.info(