Skip to content

feat: backtrader-agent P0 runtime + reproducibility hardening - #1

Merged
cloudQuant merged 6 commits into
masterfrom
feat/p0-runtime-and-hardening
Aug 1, 2026
Merged

feat: backtrader-agent P0 runtime + reproducibility hardening#1
cloudQuant merged 6 commits into
masterfrom
feat/p0-runtime-and-hardening

Conversation

@cloudQuant

Copy link
Copy Markdown
Owner

Summary

Introduces the backtrader-agent P0 runtime and a hardening pass that makes the
documented acceptance evidence reproducible in a fresh checkout and in CI. Before
this PR the repo held only .gitignore + LICENSE; the product had never entered
git, and the IMPLEMENTATION_REPORT's "59 passed / acceptance passed" claims did not
reproduce (15 tests failed out of the box).

What's included

Layered, offline-first, deterministic Backtrader strategy-authoring runtime:

  • Python runtime (src/backtrader_agent): typed CLI, opaque root registry,
    immutable SHA-256 dataset CAS (six offline adapters), canonical StrategySpec,
    package-owned corpus snapshot + 14 current-fork scaffolds, import-free
    AST/security validator, hash-bound apply/run approvals, fixed-profile
    child-process runner, append-only session hash chain with recovery, independence
    audit, create-only native adapter installer.
  • Native host adapters: claude-code, codex, opencode, openclaw.
  • References, dev scripts, typed test suite + acceptance matrix.

Hardening pass (the four gaps closed)

  1. Manifest freshness. manifest.json was stale (missing LICENSE/.gitignore;
    72 vs 74 files) and the package resources/distribution-manifest.json drifted on
    every source edit, failing the independence audit. Added scripts/build_manifest.py
    as the single regeneration entrypoint (package manifest written before root, since
    the root hashes the package manifest; verified idempotent) and a CI gate that
    committed manifests must match a fresh build.
  2. Engine-root resolution. All 14 end-to-end cells failed because
    BACKTRADER_AGENT_ACCEPTANCE_ENGINE_ROOT defaulted to the repo grandparent (rarely
    a valid Backtrader source root). New resolve_acceptance_engine_root() checks env
    var → sibling backtrader/back_trader checkouts → installed backtrader
    package, with actionable guidance when none is found. Version assertion changed
    from hardcoded 1.3.0 to the descriptor's actual version, making the suite
    portable across Backtrader builds.
  3. Listing commands + doctor engine detection. Added data list, session list,
    runs list, engine --list; doctor now reports registered engine roots and a
    hint when none is registered.
  4. Docs. Documented renderer scope (P0 renderer is a scaffold selector; entry/
    exit/sizing/risk are validated and hashed but not translated to executable
    logic), plus engine-root auto-resolution. Added CI, examples/, SECURITY.md,
    CONTRIBUTING.md, CHANGELOG.md; .gitignore now ignores .backtrader-agent/.

Verification (run locally on this branch)

Gate Result
pytest tests 59 passed
scripts/audit_independence.py passed (6/6 checks)
scripts/doctor.py ready
scripts/run_acceptance.py passed: 14 cells × 2 modes, 6 adapters, crash/resume + repair gates, clean-wheel execution, sibling-absent enforcement
manifest freshness committed manifests match build_manifest.py output

Test plan

  • Local: pytest tests, audit_independence, doctor, run_acceptance all green
  • Manifest freshness: second build_manifest.py run produces no diff
  • Smoke: data list / session list / runs list / engine --list / doctor
    return correct results on empty and populated state
  • CI green on Python 3.9 / 3.11 / 3.12 (pending first CI run)
  • Reviewer confirms examples/ walkthrough is followable end-to-end
  • Reviewer confirms no sibling product dependency (backtrader_mcp /
    backtrader_skills absent in clean install)

Notes

  • The project baseline was never in git, so baseline and hardening are
    indistinguishable in the tree; organized into 3 area-based conventional commits
    (feat runtime, chore CI/governance, docs README/report).
  • No live broker, no network data, no OS-sandbox claim — see SECURITY.md and the
    "Honest P0 limits" in README.

Offline-first, deterministic runtime for authoring and controlled
backtesting of Backtrader strategies. Layers:

- Python runtime (src/backtrader_agent): typed CLI, opaque root registry,
  immutable SHA-256 dataset CAS with six offline adapters, canonical
  StrategySpec, package-owned corpus snapshot + 14 current-fork scaffolds,
  import-free AST/security validator, hash-bound apply/run approvals,
  fixed-profile child-process runner, session hash chain with recovery,
  independence audit, and create-only native adapter installer.
- Native host adapters (claude-code, codex, opencode, openclaw).
- References, dev scripts, and the typed test suite + acceptance matrix.

Includes the P0 hardening pass: data/session/runs/engine listing commands,
doctor engine detection, scripts/build_manifest.py for exact manifest
regeneration, and a portable engine-root resolver so the 14-cell acceptance
matrix reproduces in a fresh checkout without manual env setup.
- .github/workflows/ci.yml: Python 3.9/3.11/3.12 matrix running unit tests,
  independence audit, doctor, manifest freshness, and the acceptance matrix.
- examples/: offline CSV, DataSpec, StrategySpec, and an end-to-end walkthrough.
- SECURITY.md: enforced properties and known sandbox limits.
- CONTRIBUTING.md: dev setup, manifest-regeneration duty, independence rules.
- CHANGELOG.md: P0 release and the hardening pass.
- .gitignore: ignore the .backtrader-agent/ runtime state root.
README documents the adapter/payload/runtime layering, the install and P0
workflow, sessions/recovery, reports/provenance, verification, and honest
P0 limits. Includes the engine-root auto-resolution note and the renderer
scope note (entry/exit/sizing/risk are validated but not translated to
executable logic at P0).

IMPLEMENTATION_REPORT records the P0 implemented scope, public contract
migration, enforced security properties, deferred work, and acceptance
evidence.
CI failed on all 14 end-to-end cells with BTAG-RUN-FAILED: the generated
runner imports pandas at module load (Pandas adapters + canonical feed
assembly), but `pip install backtrader` does not always pull pandas, so the
controlled child process died on import. Reproduced locally in a clean venv
without pandas; adding pandas makes the cell pass.

- pyproject [test] extras: add pandas>=1.0
- ci.yml: pip install backtrader pandas jsonschema pytest
- CONTRIBUTING.md: dev install includes pandas, with a note
- regenerate root distribution manifest
CI failed on Linux with BTAG-RUN-FAILED while macOS passed. Root cause:
RLIMIT_AS=2GB was applied on Linux (where setrlimit succeeds) but not on
macOS (where setrlimit raises and is swallowed). Scientific-Python BLAS
libraries (numpy/OpenBLAS) reserve large virtual regions on Linux that
exceed a fixed address-space budget while resident memory stays small, so
the controlled child was killed on import. Reproduced as OS-specific
(macOS venv with the same PyPI backtrader+pandas passes).

Drop RLIMIT_AS; keep RLIMIT_CPU and RLIMIT_FSIZE. The wall-clock timeout
and RLIMIT_CPU remain the real runaway guards. This is defense in depth,
not an OS sandbox (unchanged disclaimer). Regenerate distribution manifests.
Two remaining CI failures on stock Backtrader 1.9.78 (the fork 1.3.0 hid both):

- multi_indicator_system raised ZeroDivisionError in RSI's _once_op: the
  price fixture was strictly monotonic (close = 100 + index), so RSI's
  average loss was zero and stock Backtrader divided by zero in runonce.
  Oscillate the fixture (alternating up/down bars around the trend) so RSI
  has non-zero average loss; more realistic and version-agnostic.
- test_built_wheel_* failed with "invalid command 'bdist_wheel'" on Python
  3.12: the wheel test builds with --no-build-isolation, which needs
  setuptools+wheel in the env. setup-python bundles them on 3.9/3.11 but
  not 3.12. Install setuptools+wheel explicitly in CI and CONTRIBUTING.

Validated: full suite (59 tests) passes on stock Backtrader 1.9.78 in a clean
venv. Regenerate distribution manifest.
@cloudQuant
cloudQuant merged commit adc81a9 into master Aug 1, 2026
3 checks passed
@cloudQuant
cloudQuant deleted the feat/p0-runtime-and-hardening branch August 1, 2026 14:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant