docs: the example config names the model the project actually runs

[bbertucc]

Closes #176.

config.example.yaml is the file an operator copies to stand Iris up, and it pinned Opus 4.7 on both providers. The reference deployment serves every capability with Sonnet 4.6, and per #176's diagnostics sweep has done so in every benchmark round on record. So the example configured a model this project does not run and has never measured — at 1.67× the per-page cost of the figures it publishes, for an operator who changed nothing.

What changed

where	was	now
config.example.yaml openrouter default_model + per_capability ×3	anthropic/claude-opus-4.7	anthropic/claude-sonnet-4.6
config.example.yaml bedrock default_model + per_capability ×3	us.anthropic.claude-opus-4-7	us.anthropic.claude-sonnet-4-6
prd.md §10.3 example block	anthropic/claude-opus-4.7, anthropic.claude-opus-4-7-v1	Sonnet 4.6, us.-prefixed

The note, and where it lives

One note, above the first provider block, with the bedrock block pointing at it — the file's existing idiom (See the note on openrouter.max_tokens above) and the rule that a model fact belongs in exactly one place, since the model gets swapped.

What it says, beyond "this is what we run": max_tokens is our setting and not the model's, and the overrun rate, the correction shrink floor and the section headroom were all sized against how this model behaves at that ceiling. An operator on Opus 4.7 is outside those measurements and nothing in the file said so. It also notes that the choice moves the upload limits the deployment publishes, since the long edge follows the model generation — so a copy of this file now accepts exactly what the reference deployment accepts.

Judgment calls

The commented per_agent examples stay — they teach the mechanism, not a model. But page: { model: anthropic/claude-sonnet-4.6 } # cheaper/faster extraction becomes a no-op the moment Sonnet is the default, and a commented example that overrides nothing teaches nothing. It now illustrates the stronger direction, and table: takes over demonstrating the { provider:, model: } form so all three mechanisms still have a line.

providers.default stays openrouter against the deployment's bedrock, which #176 left to the maintainer. It decides which credential a new operator must obtain, and one API key with no cloud account is the shorter path to a first working run. The drift is real, so the file now states the reason rather than leaving the reader to find the mismatch.

PRD §10.3 is illustration and now says so. It pins openai/gpt-5 for one capability and names agents (image_analysis) that are not dispatched, so treating it as a recommendation was never right; it points at config.example.yaml as the file to copy and the one place that recommends. Its Bedrock id also loses the bare anthropic. prefix that config.example.yaml's own comment says is rejected for on-demand use — a copied example that 400s on the first call is worse than one naming the wrong model.

Verification

loadConfig("config.example.yaml") accepts the file and resolves all eight ids (run against a throwaway probe, since nothing in the suite loads this file):

default: openrouter
openrouter: anthropic/claude-sonnet-4.6 {"vision":...,"structured_output":...,"text":...}
bedrock: us.anthropic.claude-sonnet-4-6 {"vision":...,"structured_output":...,"text":...}

npx tsc --noEmit clean · npm test 765 pass / 0 fail · bash test/e2e.sh ALL ENDPOINTS PASSED ✅

Not changed: docs/API.md already named Sonnet 4.6 in both places, and test/image-limits.test.ts's comment about the example recommending a stronger model for verification stays true — the feedback: override is untouched.

🤖 Generated with Claude Code

[claude]

Docs and example-config only — no src/, agents/ or .github/ changes — and all six checks pass. The model swap is correct and internally consistent; two notes below, neither blocking.

Verified rather than assumed:

• All eight new ids parse via modelGeneration() to {major:4,minor:6}: at or above CACHING_FROM_GENERATION {3,7} so prompt caching stays on (src/providers/promptCache.ts:45,182), and below HIGH_RES_FROM_GENERATION {4,7} so longEdgeFor() returns 1568 (src/providers/imageLimits.ts:55-57). No other generation-gated behaviour flips for an operator who copies the file.
• No stale opus reference left in prd.md, README.md, docs/ or agents/. docs/API.md:365,386 already publish 1568 px / Sonnet 4.6, so the example now agrees with the published docs instead of contradicting them — the old file made GET /v1/limits publish 2576 px for a copy of it.
• Bedrock ids keep the us. prefix the file's own comment (config.example.yaml:272-277) says is required for on-demand use, and prd.md:1108 drops the bare anthropic. form.
• test/image-limits.test.ts:136 ("the config example recommends running verification on a stronger model than extraction") is now true rather than a no-op — the commented feedback: override was Opus 4.7 against an Opus 4.7 default before.

Non-blocking notes

1. config.example.yaml:183-185 overstates the change on the accept side.

```

tuned for. It also moves the upload limits this deployment publishes, since the

long edge follows the model generation (see image_limits below), so a copy of

this file now accepts exactly what the reference deployment accepts.

```

The long edge is advice, not a gate — line 219 of this same file says `max_long_edge_px: 1568 # published as advice; nothing is rejected for it`, and `src/providers/imageLimits.ts:47-52` says "Not an error, and deliberately not enforced". The two numbers an upload is actually rejected by are `max_image_bytes` (3,932,160, derived from the 5 MB base64 cap both providers in `BASE64_CAP_BY_PROVIDER` share) and `max_dimension_px` (8000, model-independent), and both are byte-for-byte identical before and after this diff. What moved is what `GET /v1/limits` and the upload hint publish: 2576 px → 1568 px. "publishes" is exact where "accepts" claims something the code does not do. Latent by construction — it is a comment, and the direction of the real change (advice down to the honest number) is the right one.

2. Nothing in the suite parses config.example.yaml.

Grep finds only README.md:71 and CONTRIBUTING.md:47 (cp config.example.yaml config.yaml) — which is why this PR's verification had to be a throwaway probe, as the description says. loadConfig already takes a path (src/config.ts:608), so a test that calls loadConfig(\"config.example.yaml\") and asserts resolveAgentModel for the dispatched agents would make that probe permanent; without it, an example config that is schema-invalid or has an unexpandable ${...} ships silently, and the operator hits it on their first run. A new test file also has to be registered in the test script in package.json or it never runs.

Accessibility impact: none — no code, agent prompt or output path changes; the only user-visible effect on a fresh copy of the example is that GET /v1/limits and the upload hint now publish the honest 1568 px long edge instead of 2576 px, which is advice about downscaling and rejects nothing.

[bbertucc]

Both notes taken.

1. "accepts" was wrong — thanks. The long edge is advice and rejects nothing; the two numbers an upload is actually refused by (the shared byte cap and the model-independent pixel ceiling) are identical before and after this diff. The comment now says the long edge is what GET /v1/limits and the upload hint publish, and says explicitly that it rejects nothing and that the real gates don't follow the model.

2. The example config is now parsed by the suite — test/config-example.test.ts, three tests:

• it loads, and needs exactly the one credential its default names: the unset case asserts the failure names OPENROUTER_API_KEY, and it runs first on purpose, because loadConfig memoizes by resolved path and a successful load would otherwise be handed back to it.
• every dispatched agent × every capability resolves to a model on the provider the file names.
• both blocks name one model, in the two id spellings the providers use — asserted as internal agreement rather than against a model name, since the name changes and the agreement is the property The example config and the PRD name Opus 4.7; every deployment and every benchmark number is Sonnet 4.6 #176 was about. It also pins the two silent failure modes you identified from the other direction: an id modelGeneration cannot read (falls back to the oldest generation's published long edge) and one outside the cache-eligible generations (turns every cached prefix into a full-price prefix, and only the bill says so).

Mutation-checked rather than assumed — pointing one of the eight ids at Opus 4.7 fails the run: the example names more than one generation: 4.6, 4.6, 4.6, 4.6, 4.6, 4.7, 4.6, 4.6.

One correction on the note: no package.json change was needed. The test script is node --test "test/*.test.ts", a glob, so a new file in test/ runs by being there — which is also why the count moved from 765 to 768.

tsc clean · 768 pass / 0 fail · e2e ALL ENDPOINTS PASSED ✅

[claude]

All six checks pass. The real delta against main is config.example.yaml, prd.md §10.3, and test/config-example.test.ts — the src/pipeline/lint.ts / review.ts / test/lint-nodes.test.ts hunks in my context are #197, already merged as 9c499f0 (the diff base 5c0ef27 predates it), so I have not re-reviewed them here.

Both notes from my review of 9651f24 are addressed:

• config.example.yaml:183-187 now says what the change does: "moves the long edge ... PUBLISH" plus "Only what is published: the long edge is advice about downscaling and rejects nothing, and the two limits an upload is actually refused by do not follow the model at all." That is exact.
• Nothing parsed the example; test/config-example.test.ts now does.

Withdrawing one half of that second note: I wrote "A new test file also has to be registered in the test script in package.json or it never runs." That is wrong for this repo — package.json:14 is node --test "test/*.test.ts", a glob. There was nothing to register.

Verified rather than assumed: the example carries 9 ${...} placeholders and only OPENROUTER_API_KEY is fatal when unset, because validateConfig checks only reachable providers (src/config.ts:349-351) and an unset variable expands to "" — so test 1's expected throw and test 2's successful load with one variable are both right, and the bedrock block is correctly not credential-checked. No stale opus id left in prd.md, README.md or docs/.

Non-blocking notes

1. The new test asserts internal agreement, not agreement with the numbers the docs publish.

test/config-example.test.ts checks one generation, one family, both id spellings, and cache eligibility across all eight ids. What it does not check is the one number a model swap moves outward. docs/API.md:365 ("1568 px on Sonnet 4.6") and docs/API.md:386 ("max_long_edge_px": 1568) are hardcoded, so a future bump of the example to a generation at or above HIGH_RES_FROM_GENERATION {4,7} makes GET /v1/limits publish 2576 for a fresh copy of the example while those two lines still say 1568 — the same doc-versus-deployment drift #176 was about, and all three new tests stay green through it (same generation, same family, cacheable, both prefix forms valid). One line — resolveImageLimits(loadConfig(EXAMPLE)).max_long_edge_px asserted against what docs/API.md prints — closes it. Latent: today the example and the docs agree.

2. test/config-example.test.ts:43-53 is vacuous while per_agent: {}.

const { provider, model } = resolveAgentModel(providers, agent, capability);
assert.equal(provider, "openrouter", `${agent}/${capability} left the default provider`);

resolveAgentModel (src/providers/index.ts:35-41) returns cfg.default plus per_capability[capability] ?? default_model for any agent string when there is no override, so the AGENTS list — including whether copy_editor and builder are spelled the way the dispatcher spells them — is not what this exercises; a typo or a renamed agent passes identically. What actually makes it green is test 3's "both blocks fully populated". Worth keeping as a guard for the day per_agent is non-empty, but the comment above it ("so a resolution hole cannot hide behind an agent that happens not to ask for vision") claims more than the assertions can see.

3. config.example.yaml:164 and :170 now recommend against each other.

# page: { model: anthropic/claude-opus-4.7 }     # stronger model, same provider
...
# Run the Feedback Agent (VERIFY/CLASSIFY/TRAIN) on a DIFFERENT or stronger
# model than the page agent ... (recommended):
# feedback: { model: anthropic/claude-opus-4.7 }

Both examples are the same id, so an operator who follows the page: line and the recommended feedback: line lands with verification on the same model as extraction — the thing that prose says not to do, and the arrangement test/image-limits.test.ts:136 leans on ("the config example recommends running verification on a stronger model than extraction"). Before this PR the pair read Sonnet/Opus and was consistent. Latent: both lines are comments and neither is uncommented by default; naming a third model on one of them, or dropping anthropic/claude-opus-4.7 from the page: line back to a same-generation alternative, restores the pairing.

Accessibility impact: none — no code, agent prompt or output path changes; the only user-visible effect on a fresh copy of the example is that GET /v1/limits and the upload hint publish the honest 1568 px long edge instead of 2576 px, which is advice about downscaling and rejects nothing.

[bbertucc]

One piece of outside evidence for the claim this PR turns on, and one measurement that corroborates the reviewer's note 1. Not a defect report; docs and an example config, and I would take it as it stands.

The deployment agrees, and says so without being asked

config.example.yaml:172-190 rests on "the reference deployment has served every capability with it", checked by hand and dated. That check is externally observable, because resolveImageLimits derives the published long edge from the resolved vision model (imageLimits.ts:273, longEdgeFor(resolved.model)) and GET /v1/limits prints it. Just now, against https://iris.equalify.uic.edu/v1/limits — unauthenticated, no model call, so free:

"max_long_edge_px": 1568

longEdgeFor gives 1568 for {major:4,minor:6} and 2576 from HIGH_RES_FROM_GENERATION {4,7} up (measured: anthropic/claude-sonnet-4.6 and us.anthropic.claude-sonnet-4-6 → 1568; …opus-4.7, …opus-4-7, claude-opus-5 → 2576). So the live deployment is publishing the 4.6-generation number, not the 4.7 one — which is what the example now names, and not what it named before. Caveat, because this is corroboration and not proof: the signal is generation-level rather than a model name, and an explicit image_limits.max_long_edge_px in the deployment's own config would mask it either way.

That also puts a floor under the reviewer's suggested one-liner. Asserting resolveImageLimits(loadConfig(EXAMPLE)).max_long_edge_px against what docs/API.md prints ties the example to the docs; the same number happens to be readable from the running service, so the tie can reach past this repo to the deployment — which is the one direction #176's drift ran and the one nothing in a test suite can currently see.

Note 1, measured

The reviewer's point that the three new tests are insensitive to which model the example names holds concretely, not just prospectively. I ran this PR's test/config-example.test.ts unchanged against the config.example.yaml from 5c0ef27c — the file as #176 reports it, all eight ids Opus 4.7:

✔ the example config an operator copies loads, and needs exactly the one credential it names
✔ every agent the example dispatches resolves to a model, on the provider it names
✔ both provider blocks in the example name the same model
ℹ pass 3  fail 0

Uniformly wrong is internally agreed, so the property holds. Worth saying because the comment at test/config-example.test.ts:59-61 reads as though the suite now prevents #176's silence — the file's own dated hand-check is what prevents it, and that is the honest arrangement, just not the one the comment describes. Everything else the test buys is real and I would keep all of it: an unexpanded credential failing at startup by name, both per_capability blocks fully populated rather than silently falling back, the bedrock inference-profile prefix, and cache eligibility — that last one being the failure that shows up only on the bill.

Measured with loadConfig, resolveAgentModel, modelGeneration, longEdgeFor and resolveImageLimits from 72e209d, run in a scratch copy; the second block is this PR's own test file over the previous example config, nothing patched. No model calls.

[bbertucc]

All three notes addressed in 3rd commit.

1. The long edge, tied to the docs. Done as one assertion, though not hardcoded — resolveImageLimits(loadConfig(EXAMPLE)).max_long_edge_px is checked against the number parsed out of docs/API.md, so it is a drift check rather than a third copy of 1568:

const printed = docs.match(/"max_long_edge_px":\s*(\d+)/g) ?? [];
assert.equal(printed.length, 1, `docs/API.md prints max_long_edge_px ${printed.length} times`);
assert.equal(Number(printed[0]!.match(/(\d+)/)![1]), resolved);

The count assertion is there because a second sample block added later would otherwise drift unwatched. The prose at docs/API.md:365 quotes the same figure beside the model's name and is the other half a model change has to touch; that one I left to the reader, since pinning it would tie this test to a model name, which is exactly what the #176 test deliberately does not assert.

2. The vacuous loop. The comment overclaimed and now says what the loop does: with per_agent: {} every pair walks the same fallback chain, so it cannot catch a hole hiding behind an agent that never asks for vision. What it does catch — a provider the file names but does not define, an agent the file's own comment lists that resolution has no route for, and the first thing an operator uncommenting a per_agent line hits — is worth keeping, so the loop stayed and the claim shrank.

3. The collapsed pairing. You were right that this undid the advice sitting three lines under it. Fixed by changing the page: line rather than by naming a third model, because the third form of the override syntax — the bare <agent>: <provider> shorthand the comment above documents — had no example at all:

# page: bedrock                                  # one agent on the other provider, at ITS model
# table: { model: anthropic/claude-opus-4.7 }     # stronger model, same provider
# reader: { provider: bedrock, model: us.anthropic.claude-haiku-4-5-20251001-v1:0 }

So page: keeps the default model, feedback: is the only Opus recommendation, and test/image-limits.test.ts:136's premise holds again. I added a sentence to the feedback recommendation saying why the page: line routes rather than upgrades, so the next edit to it knows it is load-bearing.

Gates: npx tsc --noEmit clean, npm test 769/769, bash test/e2e.sh → ALL ENDPOINTS PASSED ✅.

[claude]

All six checks pass. The delta since 72e209d is only config.example.yaml:164-172 and the fourth test in test/config-example.test.ts — the src/pipeline/lint.ts / review.ts / test/lint-nodes.test.ts material in my context is still #197 (merged as 9c499f0; diff base 5c0ef27 predates it), so it is not re-reviewed here.

Both notes from my 72e209d review are addressed:

• Note 1 (the test asserted internal agreement, not the number the docs publish) — test/config-example.test.ts:110-127 now derives resolveImageLimits(loadConfig(EXAMPLE)).max_long_edge_px and asserts it against the figure docs/API.md prints. Verified this is a live check and not a tautology: the example's image_limits block is commented out (config.example.yaml:220-223), so normalizeOverride(override?.max_long_edge_px, longEdgeFor(model)) (src/providers/imageLimits.ts:273) falls through to the model-derived value, and a bump to a generation at or above HIGH_RES_FROM_GENERATION would make it 2576 against the docs' 1568 and fail.
• Note 3 (page: and feedback: recommended the same model) — the page: example is now page: bedrock, and the bare-string form is real, not aspirational: agentOverride (src/providers/index.ts:14-18) reads a string entry as { provider }, and validateConfig (src/config.ts:388-405) adds it to referenced and requires only default_model for bedrock, which the file sets. So an operator who used that line gets the routing it advertises, and the feedback: Opus recommendation no longer reads against the page: line.

Non-blocking notes

1. Uncommenting any of the per_agent examples is a YAML parse error. config.example.yaml:163-172:

  per_agent: {}
    # page: bedrock                                  # one agent on the other provider, at ITS model

The examples are indented as children of a key whose value is already {}, so removing the # gives yaml (the parser loadConfig uses) All mapping items must start at the same column — the operator has to also delete the {}, which nothing says. Measured against yaml@^2.6.0 from package.json:25. Pre-existing shape rather than something this PR introduced, and this push is the one that makes the page: line worth copying — the new prose at :169-171 ("the page: line above routes rather than upgrades") points a reader straight at it. It fails loudly at startup with the line number, so nobody ships on it; it is a first-run papercut on the file whose whole job is the first run.

2. The drift test covers the JSON sample, not the two prose copies beside it. test/config-example.test.ts:123-126 asserts on docs/API.md:386 (\"max_long_edge_px\": 1568), and its printed.length === 1 guard is what keeps that anchor honest. docs/API.md:365 ("1568 px on Sonnet 4.6, 2576 px …") and :367 ("Re-saving a 12-megapixel scan at 1568 px") hardcode the same number and a model name, and neither is checked — line 365 additionally names the model, which is the fact #176 was about. In practice the failure points at API.md, so whoever bumps the model is already in the right file; the test comment ("The prose above the sample quotes the same figure beside the model's name") acknowledges this rather than asserting it. Latent today: all three agree.

Accessibility impact: none — no code, agent prompt or output path changes; the only user-visible effect on a fresh copy of the example is that GET /v1/limits and the upload hint publish the honest 1568 px long edge, which is advice about downscaling and rejects nothing.

[bbertucc]

Closing my own note from #198 (comment), which said the suite as it then stood could not catch the failure mode: the 3rd commit's assertion can.

Ran the four tests at fad236d1 in a scratch copy, once as-is and once with config.example.yaml swapped back to its pre-#198 contents and everything else left at this head:

as-is                     4/4 pass
pre-#198 example config   3 pass, 1 fail
  the long edge the example publishes is the one the API sample prints
  AssertionError: 1568 !== 2576   (docs print 1568, that config resolves 2576)

So the drift is caught in the shape it actually occurred in — a model in the example config that no longer matches the number the docs sample prints — and it is caught without a second copy of 1568 living in the test. That was note 1's point and it is answered. No further comment from me on this PR; no model calls.

[bbertucc]

Both notes fixed in the 4th commit. The first was a real defect and I had made it worse, so thank you for measuring it rather than eyeballing the indentation.

1. The uncommentable examples. Confirmed against yaml@2 before changing anything:

old shape rejected: All mapping items must start at the same column at line 3, column 1
new shape: parsed ok

The block is now commented out key and all, with the active per_agent: {} above it and a line saying to delete that line too. And the operator's instruction is now executed rather than trusted — the new test finds the commented block, strips the # , drops the {} line, writes the result to a temp path and loadConfigs it, asserting all three override forms resolve:

line	asserts
page: bedrock	routed to bedrock, at bedrock's own default_model (us. prefixed)
table: { model: … }	model pinned, provider stays the default one
reader: { provider, model }	both at once
feedback: { model: … }	claudeFamily differs from page's — the pairing the block recommends

That last row is the guard against note 3 from your previous review coming back: whichever models the example names, verification cannot land on the family extraction uses without failing the suite.

2. The prose copies. Added, for the figure only:

const sentence = docs.match(/long-edge limit \((\d+) px on /);
assert.equal(Number(sentence[1]), resolved, "the prose long edge and the configured model disagree");

The model name at docs/API.md:365 I left unpinned on purpose — asserting a name is the thing #176 asked this file to stop doing, and the number failing already lands whoever bumps the model in the right paragraph. :367 ("Re-saving a 12-megapixel scan at 1568 px") is advice phrased around the same figure and is covered by the same edit landing in that file.

Gates: npx tsc --noEmit clean, npm test 770/770, bash test/e2e.sh → ALL ENDPOINTS PASSED ✅.

Merging once the review lands on this commit.