Skip to content

docs: tighten atlas–guide routing and fix starter-API/eval-budget claims #192

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ethancjackson merged 1 commit into from
Aug 20, 2026
Merged

docs: tighten atlas–guide routing and fix starter-API/eval-budget claims #192

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 55 additions & 9 deletions docs/architecture-atlas.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -237,6 +237,7 @@
.legend-note { font-size: 14px; color: var(--ink-2); margin-top: 14px; max-width: 100ch; }
.cmp { width: 100%; }
.cmp td:first-child { color: var(--ink-2); font-family: "IBM Plex Mono", ui-monospace, monospace; font-size: .86em; text-transform: uppercase; letter-spacing: .06em; white-space: nowrap; }
.chooser td:first-child { font-weight: 600; white-space: normal; }

.agent-grid { display: grid; grid-template-columns: 1fr 1.5fr 1fr; gap: 20px; margin-top: 34px; }
.agent-grid .card { display: flex; flex-direction: column; gap: 9px; }
Expand Down Expand Up @@ -285,6 +286,14 @@
figure svg { min-width: 720px; }
.field { flex-wrap: wrap; }
.field span { text-align: left; }
.chooser, .chooser tbody, .chooser tr, .chooser td { display: block; width: 100%; }
.chooser thead { display: none; }
.chooser tr { padding: 12px 0; border-bottom: 1px solid var(--line-soft); }
.chooser td { border-bottom: none; padding: 3px 0; }
.chooser td:first-child {
font-family: "IBM Plex Mono", ui-monospace, monospace; font-size: 12px;
text-transform: uppercase; letter-spacing: .06em; color: var(--ink-2); margin-bottom: 2px;
}
}
@media (max-width: 620px) {
.fam-grid { grid-template-columns: 1fr; }
Expand All @@ -298,7 +307,7 @@
<div class="wrap">
<div class="eyebrow">Vector Institute · agentic-forecasting · bootcamp atlas</div>
<h1>Agentic Forecasting Atlas</h1>
<p class="standfirst">The system architecture of the bootcamp repo, read as one continuous story: how a series and its context get fenced behind a cutoff, how predictors — from naive baselines to tool-wielding agents — consume them, how the harness scores the results and judges the reasoning, and how five reference implementations put the same machinery to work.</p>
<p class="standfirst">The system architecture of the bootcamp repo, read as one continuous story: how a series and its context get fenced behind a cutoff, how predictors — from naive baselines to tool-wielding agents — consume them, how the harness scores the results and judges the reasoning, and how five reference implementations put the same machinery to work. This atlas is the map — the step-by-step worksheets live in <a class="ext" href="https://github.com/VectorInstitute/agentic-forecasting/tree/main/guides">guides/</a>, and §06–§07 below route you to the right one.</p>
<div class="hero-legend">
<span class="tag data">data layer</span>
<span class="tag method">predictors &amp; agents</span>
Expand Down Expand Up @@ -340,7 +349,7 @@ <h1>Agentic Forecasting Atlas</h1>
<li><a href="#s04"><span class="n">04</span><span><span class="rt">The harness</span> <span class="rd">— backtest to develop, evaluate to commit</span></span></a></li>
<li><a href="#s05"><span class="n">05</span><span><span class="rt">The agent</span> <span class="rd">— anatomy of an analyst</span></span></a></li>
<li><a href="#s06"><span class="n">06</span><span><span class="rt">The implementations</span> <span class="rd">— five instantiations, one architecture</span></span></a></li>
<li><a href="#s07"><span class="n">07</span><span><span class="rt">Build your own</span> <span class="rd">— three moves, four guides</span></span></a></li>
<li><a href="#s07"><span class="n">07</span><span><span class="rt">Build your own</span> <span class="rd">— two paths, four guides</span></span></a></li>
</ol>
</nav>
</header>
Expand Down Expand Up @@ -511,7 +520,7 @@ <h3>The exception: agents leak</h3>
</div>
</div>
</div>
<div class="refs">data/cutoff.py · data/context.py · data/service.py · <a class="ext" href="https://github.com/VectorInstitute/agentic-forecasting/blob/main/guides/01-onboard-a-dataset.md">guide 01</a> — choosing released_at honestly</div>
<div class="refs">data/cutoff.py · data/context.py · data/service.py · <a class="ext" href="https://github.com/VectorInstitute/agentic-forecasting/blob/main/guides/01-onboard-a-dataset.md#choosing-released_at-honestly">guide 01</a> — choosing released_at honestly</div>
<div class="handoff"><a href="#s03">With the data fenced, the question becomes who consumes it → 03 The predictors</a></div>
</div>
</section>
Expand Down Expand Up @@ -634,7 +643,7 @@ <h3>Two protection styles in the repo</h3>
</div>
</div>
</div>
<div class="refs">evaluation/{backtest, eval, artifacts}.py · implementations/*/specs/*.yaml · the results cache is keyed by ids, never by spec contents — bump spec_id when the window changes</div>
<div class="refs">evaluation/{backtest, eval, artifacts}.py · implementations/*/specs/*.yaml · the results cache is keyed by ids, never by spec contents — bump spec_id when the window changes · <a class="ext" href="https://github.com/VectorInstitute/agentic-forecasting/blob/main/guides/02-create-an-experiment.md#step-5--understand-the-protected-eval-before-you-need-it">guide 02</a> — specs, the cache-by-id gotcha, protected eval · <a class="ext" href="https://github.com/VectorInstitute/agentic-forecasting/blob/main/guides/04-audit-your-results.md#audit-3--decompose-the-mean">guide 04</a> — don't believe the mean</div>
<div class="handoff"><a href="#s05">The harness treats an agent like any other predictor. Inside, it is anything but → 05 The agent</a></div>
</div>
</section>
Expand Down Expand Up @@ -708,7 +717,7 @@ <h3 style="color:var(--method)">Role — AgentPredictor</h3>
</div>
</div>
</div>
<div class="refs">methods/agentic/{agent_factory, predictor}.py · analyst_agent (energy, BoC) · starter_agent (#1–#4) · <a class="ext" href="https://github.com/VectorInstitute/agentic-forecasting/blob/main/guides/03-customize-agent-strategy.md">guide 03</a> — every customization lever, with a worked change for each</div>
<div class="refs">methods/agentic/{agent_factory, predictor}.py · analyst_agent (energy, BoC) · starter_agent (#1–#4) · <a class="ext" href="https://github.com/VectorInstitute/agentic-forecasting/blob/main/guides/03-customize-agent-strategy.md">guide 03</a> — every customization lever, with a worked change for each · <a class="ext" href="https://github.com/VectorInstitute/agentic-forecasting/blob/main/guides/03-customize-agent-strategy.md#which-starter-are-you-on">which starter you are on — toolbelt vs toggles</a></div>
<div class="handoff"><a href="#s06">That is the architecture in full. Here is how five real problems put it to work → 06 The implementations</a></div>
</div>
</section>
Expand Down Expand Up @@ -815,7 +824,37 @@ <h3 class="sub">What each one teaches</h3>
</svg>
<figcaption>Each implementation stands alone — start at the problem closest to yours. The staircase is a reading order, not a dependency graph.</figcaption>
</figure>
<div class="refs"><a class="ext" href="https://github.com/VectorInstitute/agentic-forecasting/tree/main/implementations">implementations/</a>{getting_started, sp500_forecasting, food_price_forecasting, energy_oil_forecasting, boc_rate_decisions} · each README is the full walkthrough · not sure where to start? <a class="chip" href="https://github.com/VectorInstitute/agentic-forecasting/blob/main/implementations/energy_oil_forecasting/99_starter_agent.ipynb">99_starter_agent.ipynb</a></div>

<h3 class="sub">Where do you land?</h3>
<div class="tablewrap">
<table class="chooser">
<thead><tr><th>Situation</th><th>Do this</th></tr></thead>
<tbody>
<tr>
<td>Need the loop to click first</td>
<td>Run <a class="ext" href="https://github.com/VectorInstitute/agentic-forecasting/blob/main/implementations/getting_started/00_environment_check.ipynb">00_environment_check.ipynb</a> in <a class="ext" href="https://github.com/VectorInstitute/agentic-forecasting/tree/main/implementations/getting_started">getting_started</a>, then §01–§04 of this page.</td>
</tr>
<tr>
<td>Have a CSV that is none of the five problems above</td>
<td><b>Path B</b> — <a class="ext" href="https://github.com/VectorInstitute/agentic-forecasting/blob/main/guides/01-onboard-a-dataset.md">guide 1</a> → <a class="ext" href="https://github.com/VectorInstitute/agentic-forecasting/blob/main/guides/02-create-an-experiment.md">2</a> → <a class="ext" href="https://github.com/VectorInstitute/agentic-forecasting/blob/main/guides/04-audit-your-results.md">4</a> (<a class="ext" href="https://github.com/VectorInstitute/agentic-forecasting/blob/main/guides/03-customize-agent-strategy.md">guide 3</a> if you add an agent). The sample dataset in guide 1 runs offline.</td>
</tr>
<tr>
<td>Want to change how an existing agent thinks</td>
<td><b>Path A</b> — that implementation's <code class="chip">99_starter_agent.ipynb</code> + <a class="ext" href="https://github.com/VectorInstitute/agentic-forecasting/blob/main/guides/03-customize-agent-strategy.md#which-starter-are-you-on">guide 3</a> (start at "which starter are you on"), then <a class="ext" href="https://github.com/VectorInstitute/agentic-forecasting/blob/main/guides/04-audit-your-results.md">guide 4</a>.</td>
</tr>
<tr>
<td>Want news / tools / adaptive strategy</td>
<td><a class="ext" href="https://github.com/VectorInstitute/agentic-forecasting/tree/main/implementations/energy_oil_forecasting">energy_oil</a>'s <a class="ext" href="https://github.com/VectorInstitute/agentic-forecasting/blob/main/implementations/energy_oil_forecasting/README.md">README</a> then its <a class="ext" href="https://github.com/VectorInstitute/agentic-forecasting/blob/main/implementations/energy_oil_forecasting/99_starter_agent.ipynb">99_starter_agent</a> — guide 3 as written is this path (<code class="chip">ToolSpec</code> toolbelt).</td>
</tr>
<tr>
<td>Want covariates / report-grounded prompting / judged categorical</td>
<td><a class="ext" href="https://github.com/VectorInstitute/agentic-forecasting/blob/main/implementations/sp500_forecasting/README.md">sp500</a> / <a class="ext" href="https://github.com/VectorInstitute/agentic-forecasting/blob/main/implementations/food_price_forecasting/README.md">food_price</a> / <a class="ext" href="https://github.com/VectorInstitute/agentic-forecasting/blob/main/implementations/boc_rate_decisions/README.md">boc</a> README respectively, then Path A on that starter (toggles; guide 3's levers are the same <code class="chip">AgentConfig</code> fields).</td>
</tr>
</tbody>
</table>
</div>

<div class="refs"><a class="ext" href="https://github.com/VectorInstitute/agentic-forecasting/tree/main/implementations">implementations/</a>{getting_started, sp500_forecasting, food_price_forecasting, energy_oil_forecasting, boc_rate_decisions} · each README is the full walkthrough</div>
<div class="handoff"><a href="#s07">Which brings it back to you → 07 Build your own</a></div>
</div>
</section>
Expand All @@ -825,9 +864,16 @@ <h3 class="sub">What each one teaches</h3>
<div class="sec-eyebrow"><span>07</span><span>Extending the foundation</span></div>
<h2>Build your own forecaster</h2>
<div class="prose">
<p>The architecture exists so that a new forecaster is always the same three moves — and each move has a step-by-step guide in the repo.</p>
<p>The architecture exists so that a new forecaster is always one of two paths — extend a reference implementation, or bring your own dataset — and every move on either path has a step-by-step guide in the repo.</p>
</div>

<h3 class="sub">Path A — extend a reference implementation</h3>
<div class="card accent-method">
<p>Closest to a problem already in §06? Skip onboarding and specs entirely. Pick your implementation, open its <code class="chip">99_starter_agent.ipynb</code>, work guide 3's levers, and close with guide 4 — no new dataset or spec needed until you want one.</p>
<a class="chip" href="https://github.com/VectorInstitute/agentic-forecasting/blob/main/guides/03-customize-agent-strategy.md#the-levers">guides/03-customize-agent-strategy.md — the levers</a>
</div>

<h3 class="sub">Path B — bring your own dataset</h3>
<div class="steps3">
<div class="card accent-data">
<div class="n" style="color:var(--data)">1 · Onboard data</div>
Expand All @@ -841,12 +887,12 @@ <h2>Build your own forecaster</h2>
</div>
<div class="card accent-method">
<div class="n" style="color:var(--method)">3 · Shape the agent</div>
<p>Persona, toolbelt, search brief, skills — every lever that changes how the agent thinks, each with a worked change, scored against the baselines.</p>
<p>Persona, toolbelt, search brief, skills — every lever that changes how the agent thinks, each with a worked change, scored against the baselines. Only needed if you're adding an agent.</p>
<a class="chip" href="https://github.com/VectorInstitute/agentic-forecasting/blob/main/guides/03-customize-agent-strategy.md">guides/03-customize-agent-strategy.md</a>
</div>
</div>
<div class="closing">
The shape of a new forecaster is always the same: <b>implement <code class="chip">Predictor</code>, declare a spec, and run <code class="chip">backtest()</code> / <code class="chip">evaluate()</code></b> against the baselines — then <b>audit the result before you believe it</b> (<a class="chip" href="https://github.com/VectorInstitute/agentic-forecasting/blob/main/guides/04-audit-your-results.md">guides/04-audit-your-results.md</a>). Everything else in this atlas exists to keep that comparison honest.
On either path, the shape of a new forecaster is always the same: <b>implement <code class="chip">Predictor</code> and run <code class="chip">backtest()</code> / <code class="chip">evaluate()</code></b> against the baselines — then <b>audit the result before you believe it</b> (<a class="chip" href="https://github.com/VectorInstitute/agentic-forecasting/blob/main/guides/04-audit-your-results.md">guides/04-audit-your-results.md</a>). Everything else in this atlas exists to keep that comparison honest.
</div>
</div>
</section>
Expand Down
Loading