feat: backtrader-agent P0 runtime + reproducibility hardening

.github/workflows/ci.yml

@@ -0,0 +1,49 @@
+name: CI
+
+on:
+  push:
+    branches: [master]
+  pull_request:
+
+jobs:
+  test:
+    name: Test (Python ${{ matrix.python-version }})
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.9", "3.11", "3.12"]
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up Python ${{ matrix.python-version }}
+        uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install backtrader pandas jsonschema pytest setuptools wheel
+
+      - name: Resolve Backtrader engine root
+        run: |
+          echo "BACKTRADER_AGENT_ACCEPTANCE_ENGINE_ROOT=$(python -c 'import backtrader, pathlib; print(pathlib.Path(backtrader.__file__).resolve().parent.parent)')" >> "$GITHUB_ENV"
+
+      - name: Unit tests
+        run: python -m pytest tests -q -p no:cacheprovider
+
+      - name: Independence audit
+        run: python scripts/audit_independence.py
+
+      - name: Doctor
+        run: python scripts/doctor.py
+
+      - name: Distribution manifest freshness
+        run: |
+          python scripts/build_manifest.py
+          git diff --exit-code -- manifest.json src/backtrader_agent/resources/distribution-manifest.json
+
+      - name: Acceptance matrix
+        run: python scripts/run_acceptance.py

.gitignore

@@ -216,3 +216,8 @@ __marimo__/
 
 # Streamlit
 .streamlit/secrets.toml
+
+# backtrader-agent runtime state root (opaque registry, CAS, sessions, runs).
+# Only this narrow directory should be added to a target repo's ignore file.
+.backtrader-agent/
+

CHANGELOG.md

@@ -0,0 +1,55 @@
+# Changelog
+
+All notable changes to `backtrader-agent` are documented here. The product
+follows the offline-first, deterministic, independence-strict contract
+described in [README.md](README.md) and [SECURITY.md](SECURITY.md).
+
+## [Unreleased]
+
+### Fixed
+
+- Distribution manifests were stale: the root `manifest.json` omitted
+  `LICENSE` and `.gitignore`, and the package
+  `resources/distribution-manifest.json` drifted after source edits. Added
+  `scripts/build_manifest.py` as the single regeneration entrypoint and a CI
+  check that committed manifests match a fresh build.
+- `test_source_distribution_manifest_covers_every_file` counted `.git/`
+  internals in a git checkout, failing `file_count`. The exclusion set now
+  includes `.git`.
+- The acceptance engine root defaulted to the repository grandparent, which is
+  rarely a valid Backtrader source root, so all 14 end-to-end cells failed in a
+  fresh checkout. Engine roots are now resolved automatically (env var, sibling
+  `backtrader`/`back_trader` checkouts, then the installed `backtrader`
+  package) with actionable guidance when none is found.
+- The end-to-end test hardcoded the engine version `1.3.0`, failing against any
+  other Backtrader. It now asserts the run manifest records the descriptor's
+  actual version, making the suite portable.
+
+### Added
+
+- Listing commands: `data list`, `session list`, `runs list`, and
+  `engine --list` enumerate registered datasets, sessions, run results, and
+  engine roots with validity status.
+- `doctor` now reports registered engine roots and a hint when none is
+  registered.
+- CI workflow (`.github/workflows/ci.yml`) running unit tests, the independence
+  audit, doctor, manifest freshness, and the acceptance matrix across Python
+  3.9/3.11/3.12.
+- `examples/` with an offline CSV, `DataSpec`, `StrategySpec`, and a walkthrough.
+- `SECURITY.md`, `CONTRIBUTING.md`, and this changelog.
+- `.gitignore` now ignores the `.backtrader-agent/` runtime state root.
+
+### Changed
+
+- Documented renderer scope: the P0 renderer maps a `StrategySpec` to one of
+  seven fixed archetype templates parameterized by `archetype`,
+  `output_profile`, and numeric defaults. The `entry`, `exit`, `sizing`, and
+  `risk` fields are validated and recorded in the spec hash but are not
+  translated into executable logic. See
+  [references/current-fork-rules.md](references/current-fork-rules.md).
+
+## [0.1.0] - 2026-07-31
+
+Initial P0 release: independent, offline-first Backtrader strategy-authoring
+agent runtime. See [IMPLEMENTATION_REPORT.md](IMPLEMENTATION_REPORT.md) for the
+implemented scope, verification evidence, and deferred work.

CONTRIBUTING.md

@@ -0,0 +1,82 @@
+# Contributing
+
+Thanks for contributing to `backtrader-agent`. This project is offline-first,
+deterministic, and independence-strict: changes must preserve the security
+model, the content-addressed manifests, and the reproducible acceptance matrix.
+
+## Development setup
+
+```bash
+python -m pip install backtrader pandas jsonschema pytest setuptools wheel
+python -m pytest tests -q -p no:cacheprovider
+python scripts/audit_independence.py
+python scripts/doctor.py
+python scripts/run_acceptance.py
+```
+
+The generated runner imports `pandas` at module load (the Pandas adapters and
+the canonical feed assembly path need it). `pip install backtrader` does not
+always pull pandas, so install it explicitly. The wheel-distribution test
+builds with `--no-build-isolation`, so `setuptools` and `wheel` must be present
+in the environment.
+
+The tests and acceptance matrix need a Backtrader engine root: a directory
+containing `backtrader/__init__.py` and `backtrader/version.py`. It is resolved
+automatically; set `BACKTRADER_AGENT_ACCEPTANCE_ENGINE_ROOT` explicitly if
+auto-resolution fails (see [README.md](README.md#verification)).
+
+## Before opening a pull request
+
+1. **Keep manifests exact.** After any source, resource, or repository file
+   change, regenerate both distribution manifests and commit the result:
+
+   ```bash
+   python scripts/build_manifest.py
+   ```
+
+   The independence audit (`scripts/audit_independence.py`) and the
+   `test_source_distribution_manifest_covers_every_file` test fail closed when a
+   manifest drifts. CI also checks that the regenerated manifests match the
+   committed files.
+
+2. **Keep tests green.** `python -m pytest tests` must pass, including the
+   14-cell acceptance matrix (7 archetypes x `single_test`/`python_bundle`,
+   each run in both `runonce` and `runnext`).
+
+3. **Preserve independence.** Do not add imports of `backtrader_mcp`,
+   `backtrader_skills`, `fastmcp`, or `mcp`. Do not read `.agents/skills`,
+   `backtrader-mcp`, or `backtrader-skills` paths. The independence audit
+   enforces this statically.
+
+4. **Do not weaken the security model.** No candidate import in the host
+   process, no dynamic execution, no `shell=True`, no live broker/store APIs,
+   and distinct apply/run approvals. See [SECURITY.md](SECURITY.md).
+
+## Code style
+
+- Many small, focused files; high cohesion, low coupling.
+- Immutable patterns: create new objects, do not mutate.
+- Handle errors explicitly with stable `BTAG-*` codes; never swallow errors.
+- Validate at boundaries; never trust external input.
+- Type hints throughout; functions small.
+
+## Commit messages
+
+Follow conventional commits:
+
+```
+<type>: <description>
+
+<optional body>
+```
+
+Types: `feat`, `fix`, `refactor`, `docs`, `test`, `chore`, `perf`, `ci`.
+
+## Adding a new archetype or adapter
+
+New archetypes/adapter formats are product-scope changes: update
+`src/backtrader_agent/contracts.py`, the renderer in `scaffold.py`, the
+validator allowlists in `validator.py`, the acceptance matrix expectations in
+`scripts/run_acceptance.py`, and the relevant schemas under
+`src/backtrader_agent/resources/contracts/`. Then regenerate manifests and run
+the full acceptance matrix.

IMPLEMENTATION_REPORT.md

@@ -0,0 +1,132 @@
+# P0 implementation report
+
+Date: 2026-07-31
+Product version: 0.1.0
+Scope root: `backtrader-agent/`
+
+## Implemented
+
+- Python `src/backtrader_agent` distribution and typed CLI/doctor/payload.
+- Opaque controlled root registry with traversal and symlink confinement.
+- Six offline adapter paths (`generic_csv`, `backtrader_csv`, `yahoo_csv`,
+  `mt5_csv`, `pandas`, and `pandas_custom_lines`), deterministic canonical
+  text materialization, immutable SHA-256 CAS, registration, preview, TOCTOU
+  check, quality diagnostics, typed resample/replay, and canonical
+  DatasetManifest. Pandas inputs are materialized text only; pickle/object
+  deserialization is rejected.
+- Canonical StrategySpec with legacy input alias migration.
+- Package-owned 1,155-record full metadata snapshot, deterministic search and
+  provenance, plus a separate 14-entry current-fork template catalog.
+- Seven archetypes × `single_test`/`python_bundle` deterministic renderer.
+- Import-free AST/current-fork/security validator. Direct `bt.Strategy`
+  subclasses do not receive a false missing-`super()` failure.
+- Private renderer-owned signed artifact records plus locally signed,
+  short-lived validation/change/run tokens with distinct kinds and continuous
+  session/spec/dataset/artifact/provenance hash bindings.
+- Two-phase expected-hash prepare/apply, atomic target writes, postimage check,
+  create/update conflict protection, and idempotency records.
+- Fixed child-process runner with exact entrypoint, `shell=False`, minimal
+  environment, timeout, POSIX quota attempts, output limit, source/data
+  re-hashing, eleven metrics, immutable JSON/Markdown/HTML reports. A registered
+  read-only engine root is content-bound during validation; the child proves
+  that `backtrader.__file__` and version resolve from that root and records the
+  relative import path in `RunManifest`.
+- AgentSessionManifest, strictly ordered append-only event hash chain, atomic
+  checkpoint, corrupt suffix isolation, legal transitions, cancel/archive, and
+  interrupted-run pause recovery.
+- Independence audit for forbidden sibling imports/reads and dynamic execution.
+- Create-only, idempotent, manifest-driven native adapter install/uninstall for
+  Claude Code, Codex, OpenCode, and an OpenClaw workspace. OpenClaw registration
+  remains an explicit user-run official CLI step and is not falsely represented
+  by a project-local `agent.json`.
+- Seven named public JSON Schemas, AgentSessionManifest with `$defs/AgentEvent`,
+  ComparisonProfile, corpus manifest, agent payload, and wheel-content test.
+- Structured 14-cell acceptance evidence. Every archetype/profile cell performs
+  separate real `runonce` and `runnext` executions and a normalized metric
+  comparison; six adapters and specialized multi-feed, multi-timeframe, and
+  custom-line data are required. The fixed acceptance builds and clean-installs
+  a wheel, runs outside the source checkout, and records wheel hash, installed
+  origin, clean import path, and source-absence evidence. Crash/resume and
+  failure/repair are separate gates against that same clean install.
+
+## Public contract migration impact
+
+The initial local draft used short internal names. Before P0 handoff it was
+migrated to the cross-product canonical surface:
+
+- StrategySpec emits `spec_version`, `output_profile`, and `run_modes`.
+- Archetypes emit `single_data_indicator`, `multi_indicator_system`, and
+  `multi_asset_allocation` instead of their earlier short names.
+- Dataset IDs changed from a 20-character display prefix to
+  `ds_` plus the complete 64-hex semantic hash.
+- ComparisonProfile and RunResult now use the shared six integer metrics
+(`bar_num`, buy/sell/win/loss counts, `trade_num`) and five float metrics
+(`final_value`, `sharpe_ratio`, `annual_return`, `max_drawdown`,
+  `return_rate`); Sharpe and annual return are nullable.
+- DatasetManifest emits the canonical top-level core; Agent CAS/policy details
+  live in `extensions.backtrader_agent`.
+- Corpus, Artifact, Validation, RunManifest, and RunResult schemas and runtime
+  output use their canonical core fields; product-specific evidence lives under
+  `extensions`.
+
+Legacy StrategySpec field/archetype aliases remain accepted on input only.
+Previously emitted short dataset IDs cannot be migrated safely because they do
+not contain the full semantic hash; re-register the original DataSpec.
+
+## Security properties actually enforced
+
+- No candidate import in the host process and no dynamic execution API.
+- No raw command/shell/callable/pytest-target action.
+- Separate apply and execute capability kinds.
+- Authenticated product-generation evidence; external drafts, forged
+  manifests, cross-session reuse, and tampered provenance records fail closed.
+- Session, spec, source, dataset, artifact, provenance record, validation,
+  environment, engine, preimage, and mode bindings.
+- Confined relative target paths and immutable private CAS.
+- Fixed child argv and cwd; no `shell=True`.
+- Stable `BTAG-*` errors intended for redacted user display.
+
+## Known limits and deferred work
+
+- P0 child-process controls are not a full sandbox and do not prove network
+  isolation.
+- Pandas/custom-line workflows must first materialize trusted tabular text;
+  pickle/object deserialization is absent. The controlled Pandas run paths use
+  the Pandas dependency installed with Backtrader.
+- Snapshot search is lexical. Source-attached full-corpus rebuild is implemented
+  for explicitly registered read-only roots; embedding search is deferred.
+- Automated fresh master/dev worktree orchestration, cancellation signals, and
+  container runner are deferred. The runtime can execute separately approved
+  engine profiles but does not create worktrees.
+- Renderer repairs are new immutable draft revisions; an autonomous patch
+  synthesizer is not included.
+- Report HTML is intentionally minimal and contains no plotting dependency.
+
+## Acceptance evidence
+
+The authoritative evidence is produced by:
+
+```bash
+python -m pytest tests -q -p no:cacheprovider
+python scripts/doctor.py
+python scripts/audit_independence.py
+python scripts/run_acceptance.py
+```
+
+Final observed results on 2026-07-31:
+
+- product tests: `59 passed`;
+- Ruff: passed;
+- Black check: passed;
+- doctor: `ready`;
+- independence audit: all six checks passed;
+- acceptance: passed, with 7 archetypes × 2 output profiles = 14 real
+  source-bound backtest cells, two execution modes per cell, six data adapters,
+  clean-wheel execution, mandatory MCP/Skills absence, and independent
+  crash/resume and repair gates;
+- repository contract/catalog/distribution audit: passed;
+- repository `make test-fast`: `2,474 passed, 1 skipped`.
+
+The four adapter layouts are covered by installer tests. OpenClaw was not
+installed on this machine, so its external registration command remains an
+explicit user-side verification rather than a claimed live-host result.