feat: backtrader-agent P0 runtime + reproducibility hardening - #1
Merged
Conversation
Offline-first, deterministic runtime for authoring and controlled backtesting of Backtrader strategies. Layers: - Python runtime (src/backtrader_agent): typed CLI, opaque root registry, immutable SHA-256 dataset CAS with six offline adapters, canonical StrategySpec, package-owned corpus snapshot + 14 current-fork scaffolds, import-free AST/security validator, hash-bound apply/run approvals, fixed-profile child-process runner, session hash chain with recovery, independence audit, and create-only native adapter installer. - Native host adapters (claude-code, codex, opencode, openclaw). - References, dev scripts, and the typed test suite + acceptance matrix. Includes the P0 hardening pass: data/session/runs/engine listing commands, doctor engine detection, scripts/build_manifest.py for exact manifest regeneration, and a portable engine-root resolver so the 14-cell acceptance matrix reproduces in a fresh checkout without manual env setup.
- .github/workflows/ci.yml: Python 3.9/3.11/3.12 matrix running unit tests, independence audit, doctor, manifest freshness, and the acceptance matrix. - examples/: offline CSV, DataSpec, StrategySpec, and an end-to-end walkthrough. - SECURITY.md: enforced properties and known sandbox limits. - CONTRIBUTING.md: dev setup, manifest-regeneration duty, independence rules. - CHANGELOG.md: P0 release and the hardening pass. - .gitignore: ignore the .backtrader-agent/ runtime state root.
README documents the adapter/payload/runtime layering, the install and P0 workflow, sessions/recovery, reports/provenance, verification, and honest P0 limits. Includes the engine-root auto-resolution note and the renderer scope note (entry/exit/sizing/risk are validated but not translated to executable logic at P0). IMPLEMENTATION_REPORT records the P0 implemented scope, public contract migration, enforced security properties, deferred work, and acceptance evidence.
CI failed on all 14 end-to-end cells with BTAG-RUN-FAILED: the generated runner imports pandas at module load (Pandas adapters + canonical feed assembly), but `pip install backtrader` does not always pull pandas, so the controlled child process died on import. Reproduced locally in a clean venv without pandas; adding pandas makes the cell pass. - pyproject [test] extras: add pandas>=1.0 - ci.yml: pip install backtrader pandas jsonschema pytest - CONTRIBUTING.md: dev install includes pandas, with a note - regenerate root distribution manifest
CI failed on Linux with BTAG-RUN-FAILED while macOS passed. Root cause: RLIMIT_AS=2GB was applied on Linux (where setrlimit succeeds) but not on macOS (where setrlimit raises and is swallowed). Scientific-Python BLAS libraries (numpy/OpenBLAS) reserve large virtual regions on Linux that exceed a fixed address-space budget while resident memory stays small, so the controlled child was killed on import. Reproduced as OS-specific (macOS venv with the same PyPI backtrader+pandas passes). Drop RLIMIT_AS; keep RLIMIT_CPU and RLIMIT_FSIZE. The wall-clock timeout and RLIMIT_CPU remain the real runaway guards. This is defense in depth, not an OS sandbox (unchanged disclaimer). Regenerate distribution manifests.
Two remaining CI failures on stock Backtrader 1.9.78 (the fork 1.3.0 hid both): - multi_indicator_system raised ZeroDivisionError in RSI's _once_op: the price fixture was strictly monotonic (close = 100 + index), so RSI's average loss was zero and stock Backtrader divided by zero in runonce. Oscillate the fixture (alternating up/down bars around the trend) so RSI has non-zero average loss; more realistic and version-agnostic. - test_built_wheel_* failed with "invalid command 'bdist_wheel'" on Python 3.12: the wheel test builds with --no-build-isolation, which needs setuptools+wheel in the env. setup-python bundles them on 3.9/3.11 but not 3.12. Install setuptools+wheel explicitly in CI and CONTRIBUTING. Validated: full suite (59 tests) passes on stock Backtrader 1.9.78 in a clean venv. Regenerate distribution manifest.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Introduces the
backtrader-agentP0 runtime and a hardening pass that makes thedocumented acceptance evidence reproducible in a fresh checkout and in CI. Before
this PR the repo held only
.gitignore+LICENSE; the product had never enteredgit, and the IMPLEMENTATION_REPORT's "59 passed / acceptance passed" claims did not
reproduce (15 tests failed out of the box).
What's included
Layered, offline-first, deterministic Backtrader strategy-authoring runtime:
src/backtrader_agent): typed CLI, opaque root registry,immutable SHA-256 dataset CAS (six offline adapters), canonical StrategySpec,
package-owned corpus snapshot + 14 current-fork scaffolds, import-free
AST/security validator, hash-bound apply/run approvals, fixed-profile
child-process runner, append-only session hash chain with recovery, independence
audit, create-only native adapter installer.
Hardening pass (the four gaps closed)
manifest.jsonwas stale (missingLICENSE/.gitignore;72 vs 74 files) and the package
resources/distribution-manifest.jsondrifted onevery source edit, failing the independence audit. Added
scripts/build_manifest.pyas the single regeneration entrypoint (package manifest written before root, since
the root hashes the package manifest; verified idempotent) and a CI gate that
committed manifests must match a fresh build.
BACKTRADER_AGENT_ACCEPTANCE_ENGINE_ROOTdefaulted to the repo grandparent (rarelya valid Backtrader source root). New
resolve_acceptance_engine_root()checks envvar → sibling
backtrader/back_tradercheckouts → installedbacktraderpackage, with actionable guidance when none is found. Version assertion changed
from hardcoded
1.3.0to the descriptor's actual version, making the suiteportable across Backtrader builds.
data list,session list,runs list,engine --list;doctornow reports registered engine roots and ahint when none is registered.
entry/exit/sizing/riskare validated and hashed but not translated to executablelogic), plus engine-root auto-resolution. Added CI,
examples/, SECURITY.md,CONTRIBUTING.md, CHANGELOG.md;
.gitignorenow ignores.backtrader-agent/.Verification (run locally on this branch)
pytest testsscripts/audit_independence.pyscripts/doctor.pyscripts/run_acceptance.pybuild_manifest.pyoutputTest plan
pytest tests,audit_independence,doctor,run_acceptanceall greenbuild_manifest.pyrun produces no diffdata list/session list/runs list/engine --list/doctorreturn correct results on empty and populated state
examples/walkthrough is followable end-to-endbacktrader_mcp/backtrader_skillsabsent in clean install)Notes
indistinguishable in the tree; organized into 3 area-based conventional commits
(
featruntime,choreCI/governance,docsREADME/report)."Honest P0 limits" in README.