ci: consolidate deterministic CI and live E2E workflows

.agents/skills/quantmind-dev/SKILL.md

@@ -30,11 +30,20 @@ A feature task usually chains all three: develop → commit → pull request.
 
 ## Rules
 
-- `bash scripts/verify.sh` is the deterministic offline golden gate. Run it
-  before every push and before marking a PR ready.
-- Public-network integrations have separate bounded live component gates.
-  Run every applicable gate listed in `docs/README.md` when changing that
-  component and before publishing.
+- `bash scripts/verify.sh` is the deterministic required verification. Run it
+  before every push and before marking a PR ready. The required
+  `.github/workflows/ci.yml` workflow runs the same harness.
+- Public-network integrations have separate bounded smoke tests.
+  `.github/workflows/e2e.yml` owns their scheduled, manual, and path-filtered
+  component jobs. Run every applicable test listed in `docs/README.md` when
+  changing that component and before publishing. External service availability
+  must not block unrelated changes.
+- Keep `docs/README.md` as the single catalog of component smoke-test commands.
+  A new live component adds a component-specific
+  `scripts/verify_<component>_e2e.py`, a named job in the existing `e2e.yml`,
+  precise PR path coverage, and one catalog row. When multiple live jobs exist,
+  use GitHub-native per-job change detection so only affected jobs run. Do not
+  add a separate workflow, generic E2E runner or registry, or base E2E class.
 - Never bypass pre-commit / pre-push hooks (`--no-verify`) unless the user
   explicitly authorizes it.
 - New features ship with a unit test and a focused example (see

.agents/skills/quantmind-dev/references/commit.md

@@ -45,7 +45,8 @@ Do not mix mechanical reformatting with behavior changes.
 
 2. Run verification appropriate to the change:
    - during development: targeted tests, e.g. `pytest tests/<module>/`
-   - before push / handoff: `bash scripts/verify.sh` (canonical gate)
+   - before push / handoff: `bash scripts/verify.sh` (deterministic required
+     verification)
 
 3. Commit. Pre-commit hooks run ruff format/check and file hygiene checks;
    the pre-push hook runs the full `scripts/verify.sh`.

.agents/skills/quantmind-dev/references/develop-components.md

@@ -21,9 +21,9 @@ apply throughout.
 5. **Update the public surface** if needed: package exports, the relevant
    design or guide, and the catalog in `docs/README.md`. Update the root
    README only when its overview or quick start changes.
-6. **Verify**: run targeted tests while iterating, the offline golden gate
-   before handoff, and every applicable live component gate for changed
-   public-network integrations.
+6. **Verify**: run targeted tests while iterating, deterministic required
+   verification before handoff, and every applicable live-network component
+   smoke test for changed public-network integrations.
 
 ## Module Routing
 
@@ -108,8 +108,15 @@ When adding a source to an existing operation:
 3. Add offline mocked tests for parsing, boundaries, continuation after item
    failures, and completeness semantics.
 4. Update the source table and design under `docs/`.
-5. Add or update a bounded live verifier and GitHub workflow. Keep live
-   network work out of `scripts/verify.sh`.
+5. Add or update a component-specific
+   `scripts/verify_<component>_e2e.py` and list its command in
+   `docs/README.md`.
+6. Add or update the component's named job in the existing
+   `.github/workflows/e2e.yml` and extend the workflow's precise pull-request
+   path filter. When multiple live jobs exist, use GitHub-native per-job change
+   detection so only affected component jobs run. Do not add a separate
+   workflow, generic runner or registry, or base E2E class. Keep live network
+   work out of `scripts/verify.sh`.
 
 ## Tests
 

.agents/skills/quantmind-dev/references/pull-request.md

@@ -7,16 +7,17 @@ How to open and maintain a pull request against QuantMind.
 1. Branch from `master`; keep the PR small and focused — one PR equals one
    reviewable change (see
    [Google eng practices on small CLs](https://google.github.io/eng-practices/review/developer/small-cls.html)).
-2. Run the deterministic offline golden gate:
+2. Run the deterministic required verification:
 
    ```bash
    bash scripts/verify.sh
    ```
 
 3. If the change affects a public-network integration, run every applicable
-   live component gate listed in `docs/README.md`.
+   live-network component smoke test listed in `docs/README.md`.
 
-Do not open or mark ready a PR with a red offline or applicable live gate.
+Do not open or mark ready a PR with red deterministic verification or an
+applicable red live-network smoke test.
 
 ## Title
 
@@ -39,10 +40,10 @@ and make sure the body covers:
    if one exists.
 2. **Related issue** — reference it when one exists (`Closes #NN` /
    `Part of #NN`).
-3. **Verification performed** — state the exact offline and applicable live
-   commands you ran (e.g. `bash scripts/verify.sh`; targeted
-   `pytest tests/<module>/`; a component verifier; manual example run) so the
-   reviewer does not have to guess.
+3. **Verification performed** — state the exact deterministic and applicable
+   live-network commands you ran (e.g. `bash scripts/verify.sh`; targeted
+   `pytest tests/<module>/`; a component-specific verifier; manual example
+   run) so the reviewer does not have to guess.
 
 Keep the template checklist and remove items that do not apply.
 

.claude/skills/quantmind-dev/SKILL.md

@@ -30,11 +30,20 @@ A feature task usually chains all three: develop → commit → pull request.
 
 ## Rules
 
-- `bash scripts/verify.sh` is the deterministic offline golden gate. Run it
-  before every push and before marking a PR ready.
-- Public-network integrations have separate bounded live component gates.
-  Run every applicable gate listed in `docs/README.md` when changing that
-  component and before publishing.
+- `bash scripts/verify.sh` is the deterministic required verification. Run it
+  before every push and before marking a PR ready. The required
+  `.github/workflows/ci.yml` workflow runs the same harness.
+- Public-network integrations have separate bounded smoke tests.
+  `.github/workflows/e2e.yml` owns their scheduled, manual, and path-filtered
+  component jobs. Run every applicable test listed in `docs/README.md` when
+  changing that component and before publishing. External service availability
+  must not block unrelated changes.
+- Keep `docs/README.md` as the single catalog of component smoke-test commands.
+  A new live component adds a component-specific
+  `scripts/verify_<component>_e2e.py`, a named job in the existing `e2e.yml`,
+  precise PR path coverage, and one catalog row. When multiple live jobs exist,
+  use GitHub-native per-job change detection so only affected jobs run. Do not
+  add a separate workflow, generic E2E runner or registry, or base E2E class.
 - Never bypass pre-commit / pre-push hooks (`--no-verify`) unless the user
   explicitly authorizes it.
 - New features ship with a unit test and a focused example (see

.claude/skills/quantmind-dev/references/commit.md

@@ -45,7 +45,8 @@ Do not mix mechanical reformatting with behavior changes.
 
 2. Run verification appropriate to the change:
    - during development: targeted tests, e.g. `pytest tests/<module>/`
-   - before push / handoff: `bash scripts/verify.sh` (canonical gate)
+   - before push / handoff: `bash scripts/verify.sh` (deterministic required
+     verification)
 
 3. Commit. Pre-commit hooks run ruff format/check and file hygiene checks;
    the pre-push hook runs the full `scripts/verify.sh`.

.claude/skills/quantmind-dev/references/develop-components.md

@@ -21,9 +21,9 @@ apply throughout.
 5. **Update the public surface** if needed: package exports, the relevant
    design or guide, and the catalog in `docs/README.md`. Update the root
    README only when its overview or quick start changes.
-6. **Verify**: run targeted tests while iterating, the offline golden gate
-   before handoff, and every applicable live component gate for changed
-   public-network integrations.
+6. **Verify**: run targeted tests while iterating, deterministic required
+   verification before handoff, and every applicable live-network component
+   smoke test for changed public-network integrations.
 
 ## Module Routing
 
@@ -108,8 +108,15 @@ When adding a source to an existing operation:
 3. Add offline mocked tests for parsing, boundaries, continuation after item
    failures, and completeness semantics.
 4. Update the source table and design under `docs/`.
-5. Add or update a bounded live verifier and GitHub workflow. Keep live
-   network work out of `scripts/verify.sh`.
+5. Add or update a component-specific
+   `scripts/verify_<component>_e2e.py` and list its command in
+   `docs/README.md`.
+6. Add or update the component's named job in the existing
+   `.github/workflows/e2e.yml` and extend the workflow's precise pull-request
+   path filter. When multiple live jobs exist, use GitHub-native per-job change
+   detection so only affected component jobs run. Do not add a separate
+   workflow, generic runner or registry, or base E2E class. Keep live network
+   work out of `scripts/verify.sh`.
 
 ## Tests
 

.claude/skills/quantmind-dev/references/pull-request.md

@@ -7,16 +7,17 @@ How to open and maintain a pull request against QuantMind.
 1. Branch from `master`; keep the PR small and focused — one PR equals one
    reviewable change (see
    [Google eng practices on small CLs](https://google.github.io/eng-practices/review/developer/small-cls.html)).
-2. Run the deterministic offline golden gate:
+2. Run the deterministic required verification:
 
    ```bash
    bash scripts/verify.sh
    ```
 
 3. If the change affects a public-network integration, run every applicable
-   live component gate listed in `docs/README.md`.
+   live-network component smoke test listed in `docs/README.md`.
 
-Do not open or mark ready a PR with a red offline or applicable live gate.
+Do not open or mark ready a PR with red deterministic verification or an
+applicable red live-network smoke test.
 
 ## Title
 
@@ -39,10 +40,10 @@ and make sure the body covers:
    if one exists.
 2. **Related issue** — reference it when one exists (`Closes #NN` /
    `Part of #NN`).
-3. **Verification performed** — state the exact offline and applicable live
-   commands you ran (e.g. `bash scripts/verify.sh`; targeted
-   `pytest tests/<module>/`; a component verifier; manual example run) so the
-   reviewer does not have to guess.
+3. **Verification performed** — state the exact deterministic and applicable
+   live-network commands you ran (e.g. `bash scripts/verify.sh`; targeted
+   `pytest tests/<module>/`; a component-specific verifier; manual example
+   run) so the reviewer does not have to guess.
 
 Keep the template checklist and remove items that do not apply.
 

.github/PULL_REQUEST_TEMPLATE.md

@@ -8,13 +8,13 @@
 
 ## Verification
 
-<!-- List the exact offline and applicable live commands you ran. -->
+<!-- List the exact deterministic and applicable live-network commands. -->
 
 ## Checklist
 
 - [ ] The title uses English Conventional Commit format: `type(scope): summary`.
 - [ ] The related issue or design discussion is linked when applicable.
 - [ ] `bash scripts/verify.sh` passes.
-- [ ] Every applicable live component gate passes, or this PR states why none applies.
+- [ ] Every applicable live-network component smoke test passes, or this PR states why none applies.
 - [ ] Public behavior has focused tests, an example, and documentation where applicable.
 - [ ] The PR is complete, small, and contains no unrelated changes.

.github/workflows/verify.yml → .github/workflows/ci.yml

@@ -1,4 +1,4 @@
-name: verify
+name: ci
 
 on:
   pull_request:
@@ -7,15 +7,17 @@ on:
     branches: [master]
   workflow_dispatch:
 
-# Cancel in-progress runs on the same PR/branch when a new commit lands.
+permissions:
+  contents: read
+
 concurrency:
-  group: verify-${{ github.workflow }}-${{ github.ref }}
+  group: ${{ github.workflow }}-${{ github.ref }}
   cancel-in-progress: true
 
 jobs:
-  verify:
+  ci:
     runs-on: ubuntu-latest
-    timeout-minutes: 15
+    timeout-minutes: 20
 
     steps:
       - name: Checkout
@@ -35,10 +37,13 @@ jobs:
       - name: Create virtual environment
         run: uv venv
 
-      - name: Install project + dev dependencies
+      - name: Install project and development dependencies
         run: uv pip install --python .venv/bin/python -e ".[dev]"
 
-      - name: Run scripts/verify.sh
+      - name: Run file-hygiene hooks
         run: |
           source .venv/bin/activate
-          bash scripts/verify.sh
+          SKIP=ruff,ruff-format pre-commit run --all-files --hook-stage pre-commit
+
+      - name: Run deterministic verification
+        run: bash scripts/verify.sh

.github/workflows/news-e2e.yml → .github/workflows/e2e.yml

@@ -1,8 +1,21 @@
-name: news-e2e
+name: e2e
 
 on:
   pull_request:
     branches: [master]
+    paths:
+      - '.github/workflows/e2e.yml'
+      - 'scripts/verify_news_e2e.py'
+      - 'quantmind/preprocess/_news_types.py'
+      - 'quantmind/preprocess/clean.py'
+      - 'quantmind/preprocess/news.py'
+      - 'quantmind/preprocess/pr_newswire.py'
+      - 'quantmind/preprocess/time.py'
+      - 'quantmind/preprocess/format/html.py'
+      - 'quantmind/preprocess/fetch/__init__.py'
+      - 'quantmind/preprocess/fetch/_types.py'
+      - 'quantmind/preprocess/fetch/http.py'
+      - 'quantmind/preprocess/fetch/rss.py'
   schedule:
     - cron: "17 3 * * *"
   workflow_dispatch:
@@ -15,7 +28,7 @@ concurrency:
   cancel-in-progress: true
 
 jobs:
-  news-e2e:
+  news:
     runs-on: ubuntu-latest
     timeout-minutes: 10
 

.pre-commit-config.yaml

@@ -19,7 +19,7 @@ repos:
 -   repo: local
     hooks:
     -   id: verify
-        name: Run scripts/verify.sh (full golden harness)
+        name: Run scripts/verify.sh (deterministic verification harness)
         entry: bash scripts/verify.sh
         language: system
         pass_filenames: false

AGENTS.md

@@ -34,17 +34,26 @@ resurrect it into master.
 ```bash
 uv venv && source .venv/bin/activate
 uv pip install -e ".[dev]"
-bash scripts/verify.sh              # deterministic offline golden gate
-python scripts/verify_news_e2e.py   # live PR Newswire component gate
+bash scripts/verify.sh              # deterministic required verification
 ```
 
 `scripts/verify.sh` runs five fast-fail steps (`ruff format --check`,
 `ruff check`, `basedpyright`, `lint-imports`, `pytest --cov`) and must remain
-network-free. Public-network integrations have separate bounded live gates;
-run each applicable gate when changing that component and before publishing.
-The current component catalog and commands live in `docs/README.md`. Do not
-bypass pre-commit / pre-push hooks unless the user explicitly authorizes it —
-fix the underlying issue instead.
+network-free. `.github/workflows/ci.yml` is the required deterministic CI
+workflow. Public-network integrations have separate bounded smoke tests;
+`.github/workflows/e2e.yml` owns their scheduled, manual, and path-filtered
+component jobs. Run each applicable smoke test when changing that component
+and before publishing. External service availability must not block changes
+outside that component. `docs/README.md` is the single catalog of component
+commands; do not enumerate them in this file.
+
+To extend live verification, add a component-specific
+`scripts/verify_<component>_e2e.py`, add a named job to the existing `e2e.yml`,
+extend its precise PR path filter, and add one catalog row. When a second live
+job is added, use GitHub-native per-job change detection so PRs run only the
+affected component jobs. Do not add another E2E workflow, a generic runner or
+registry, or a base E2E class. Do not bypass pre-commit / pre-push hooks unless
+the user explicitly authorizes it — fix the underlying issue instead.
 
 ## Architecture Constraints (stable)