feat: harness contract and plugin SPI (plan 1 of 3)

.claude/harness-candidates.md

@@ -962,3 +962,6 @@ re-derive from scratch.
   leaves them stale with nothing failing. Needs a backtick-path extractor scoped to
   one section, which is the narrow case of the prose-path candidate above. — caught
   during the reports consolidation rebase.
+- [ ] A `TaskDefinition` serialized for a later reload (docker `_stage_inputs`, Harbor `environment/task.yaml`) must dump `agent` with `exclude_unset=True`, or the reload marks model defaults as set and the harness contract check rejects the task — two round-trip tests guard today's two sites, but nothing flags a third `task.model_dump(` written for reload; needs a call-site classifier, not a name match — caught in the harness-contract final review.
+- [ ] A real-SDK Antigravity policy test: run `policy.enforce(agent._policies(real_policy))` to prove deny-beats-allow and `finish` approval against the installed SDK instead of a SimpleNamespace fake — nothing exercises the SDK's own bucket precedence; needs study of the hook-evaluation API — caught in the harness-contract Phase 3 review.
+- [ ] OpenCode: warn when an inherited `OPENCODE_CONFIG_CONTENT` `permission` / `instructions` value is not a dict / list and is replaced — today it is dropped silently; small, but needs a decision on warn vs. keep — caught in the harness-contract Phase 3 review.

.claude/notes/agents.md

@@ -82,6 +82,12 @@ intentionally brief and out of scope; trimming for DISPLAY belongs in the render
   table's to state, not this file's. Full table + rationale:
   docs/agents/HARNESS_PARITY.md.
 
+  The agent-field half of parity is now the `HarnessContract` each agent class declares:
+  a field, `permission_mode` value or tool name a harness cannot honor is a resolution
+  error, and `make parity-table` renders the contract (CE069 checks it), so the page can no
+  longer drift from the adapters. The run-limit half is still the hand-written table above;
+  Plan 2 moves it onto the contract.
+
 ## Shared turn lifecycle
 
 Every adapter drives the same skeleton, on the base class: `_begin_turn()` resets the
@@ -122,9 +128,10 @@ budget again and re-hits the cap. `ended_cleanly` is the guard.
 `create_agent` calls `agent_class(config, route=route, **kwargs)` through a
 `cast(Any, ...)`, so pyright checks nothing at the call site; a `**_` sink would mean
 nothing checks it at runtime either. The orchestrator depends on that `TypeError` as a
-signal — it gates `cost_log_tags` on `supports_cost_log_tags` precisely because the
-agent-agnostic factory would otherwise forward it into constructors that do not declare
-it. A mis-gated kwarg must be loud, not silently dropped.
+signal: a kwarg forwarded into a constructor that does not declare it must be loud, not
+silently dropped. `cost_log_tags` is declared on the base `Agent.__init__` and every
+subclass forwards it, so the factory passes it on every LiteLLM route without a
+capability gate.
 
 `route` is accepted for factory parity and deliberately unused by the CLI-driven
 harnesses: those CLIs own their own provider configuration.
@@ -473,6 +480,9 @@ shell-aware `parameters["command"]` extraction in `criteria/command_executed.py`
 to raw-JSON matching — so the same task scores differently per harness. Unknown names pass
 through unchanged.
 
+The Claude-to-native maps the uniform tool fields use are derived by inverting these, never
+written twice.
+
 Three cases are worth knowing:
 
 - **OpenCode's tool set varies by MODEL within the one harness.** A live 174-task run
@@ -493,6 +503,47 @@ any key that FIRST appears at DONE is treated as a result — which matters beca
 `skill_triggered` substring-searches every parameter value, so a leaked result could
 false-positive.
 
+## The uniform fields, per harness
+
+`permission_mode`, `allowed_tools` and `disallowed_tools` stay on `BaseAgentConfig` as one
+interface, and each harness honors them where the pinned CLI or SDK has a verified mechanism
+(spike of 2026-09-16). Their meaning is the same everywhere: an allowlist permits only the
+named tools, a deny always wins, and `plan` denies the Write, Edit and Bash equivalents
+(`READ_ONLY_DENIED_TOOLS`, one declaration for every adapter). An empty `allowed_tools: []`
+restricts nothing, because Claude Code passes `[]` as "no `--allowedTools` flag"; the same
+YAML must not mean "all tools" on one harness and "no tools" on the others.
+
+One meaning per field is not enough; each VALUE needs one too (`c/harness-architecture-comparison.md`
+§ 6, P0-1 and P0-2). Two defects made that concrete. The inverse tool maps were read with
+`.get(name, ())`, so a typo or a name the harness lacks silently restricted nothing. And
+`permission_mode: default` meant "ask for approval" on Claude Code but "run autonomously" on the
+other harnesses. So the contract lists the `permission_modes` a harness honors, and `tool_names` is a
+`ToolNameMap` that is total and closed over `CANONICAL_TOOL_NAMES`: a canonical name the harness has
+no tool for maps to `()` explicitly, and a missing row fails at adapter import. `Task` and `Agent` both
+name the subagent tool (`TOOL_NAME_ALIASES`), so `from_inverse` gives `Task` the natives of `Agent`;
+otherwise the older spelling, which the corpus still uses, would restrict nothing. Pi, OpenCode and
+Antigravity honor only `plan` and `bypassPermissions` until a native mechanism with the Claude Code
+meaning of `default` / `acceptEdits` is verified.
+
+- **Pi** (0.85.1): `--tools <csv>` is an allowlist and `--exclude-tools <csv>` a denylist over
+  the lowercase built-ins. The denied set is subtracted before `--tools` is emitted, and an
+  allowlist that maps to nothing becomes `--no-tools`.
+- **OpenCode** (1.18.30): the `permission` config accepts `"*": "deny"` plus per-key
+  `allow` / `deny`, and `--auto` approves only what is not explicitly denied — so `plan` is
+  explicit denies and `--auto` is passed on every run. `instructions` files are read by
+  `session/instruction.ts::system()` and spread into the SYSTEM messages, so
+  `system_prompt` is a temp file listed there (append). The file lives outside the sandbox
+  for the same reason as the skill paths below. The permission keys are coarser than tool
+  names (`edit` governs every write-shaped tool); that four-entry table is the one literal.
+  `"*"` also matches non-tool permissions (`external_directory`, `doom_loop`), which the CLI
+  merges before config rules and `--auto` used to approve, so an allowlist re-allows them.
+  OpenCode applies the LAST matching rule, so our rules are placed after inherited ones.
+- **Antigravity** (0.1.8): `hooks/policy.py` buckets specific rules above wildcard ones and
+  deny above allow, so rule order does not matter. `finish` is always allowed under an
+  allowlist because the harness ends a turn with it; whether `deny_all()` reaches it could
+  not be probed offline, and allowing it is the safe direction.
+- **Codex**: no mechanism (next section), so all three rows are unsupported.
+
 ## Codex runs full-access on every permission mode
 
 `coder_eval` owns the isolation boundary either way — a docker container or an ephemeral
@@ -504,10 +555,15 @@ and scores 0 with no loud error. Dropping to full-access matches claude-code and
 Antigravity, which run with no in-agent OS sandbox; it also keeps network on, so tool
 installs work without extra sandbox config.
 
-The consequence is stated loudly at `start()` for EVERY mode, not just
-`bypassPermissions`, so operators are not misled that plan/acceptEdits/default confine
-Codex — none of them do. Adversarial or untrusted evals belong on the docker driver; the
-tempdir/host driver is a working directory, not a confinement boundary.
+Codex's contract therefore marks `permission_mode` unsupported, so a Codex task that sets
+any mode is rejected at resolution rather than believing plan/acceptEdits/default confine
+it. Adversarial or untrusted evals belong on the docker driver; the tempdir/host driver is
+a working directory, not a confinement boundary.
+
+Tool restriction is not available either. `strings` on the pinned codex-cli 0.39.0 binary
+shows `enabled_tools` / `disabled_tools` only inside `RawMcpServerConfig` (beside
+`bearer_token_env_var`, `startup_timeout_sec`); there is no top-level key. The adapter's old
+top-level `config.enabled_tools` forward therefore never restricted a tool, and was deleted.
 
 Approval mode is `deny_all` on every permission mode too. The SDK offers only two:
 `auto_review`, which puts a SERVER-SIDE reviewer in the loop that can spuriously return
@@ -599,6 +655,8 @@ field. **A trend dashboard must not pool scores across that boundary**, and an a
 marker reads as a pre-marker run — which is why every adapter spreads the base
 `get_environment_info()` first rather than emitting the marker conditionally (CE046).
 
+The class default is the `system_prompt_semantics` field of the agent's `HarnessContract`
+(the base emits `"unknown"` when the contract marks `system_prompt` unsupported).
 claude-code's is the only one derived per config rather than fixed, so it is computed
 from the resolved prompt value and never recomputed independently — the persisted regime
 cannot disagree with what was sent.

.claude/notes/orchestration.md

@@ -26,6 +26,26 @@
   agent.type=…` last-win rather than hard-error (the `-D` value wins). Tools, plugins,
   and SDK options are `-D`-only.
 
+- **Per-kind defaults (`by_type`)**: an experiment's `defaults.agent` may carry
+  `by_type: {<kind>: {...}}`. Each entry is inserted as its own layer directly above the
+  experiment layer that carries it, so it stays BELOW the task. The entry is selected by the
+  FINAL kind across all five layers (`cli_agent_type` gives the CLI half), so `--type pi`
+  never inherits Claude-only values, which would otherwise be rejected by the contract
+  check. A kind that is not installed is tolerated (DEBUG log): a shared experiment YAML
+  can name a plugin kind that only some hosts have. It is not allowed on a task or a
+  variant, since both already know their kind.
+
+- **The harness contract check** is the second hard resolution-time rejection, beside
+  early stop. Both raise a `TaskResolutionError`, which `resolve_all_tasks` re-raises
+  instead of demoting to a skipped task and `plan` turns into a non-zero exit. A field
+  counts as SET only if a layer wrote it with a non-null value, so a model default and
+  the default experiment's `plugins: null` never trip it. For a set field on an enforcing
+  harness it also checks the VALUE: a `permission_mode` outside `contract.permission_modes`
+  and a tool name outside `CANONICAL_TOOL_NAMES` are rejected, because the adapters index
+  their total `ToolNameMap` and would otherwise meet the name mid-run. A `-D
+  agent.system_prompt_file` is inlined against `Path.cwd()` after layer 5, so no adapter
+  and no container mount ever sees a prompt file.
+
 ## Execute vs. run: the grading switch
 
 - **Execute vs. run (the grading switch)**: `coder-eval execute` is `coder-eval run`

.claude/notes/reporting.md

@@ -82,10 +82,11 @@ right after the counter bump at the top of `communicate()` and consumed by
 when partial-record assembly leaves `pending_turn` at None. That is why rollback is the
 caller's move, not the agent's: only the caller knows a turn failed.
 
-Capability flags are declared rather than probed. `supports_cooperative_stop` gates
-arming early-stop, so arming it on an agent that ignores `should_stop` is rejected at
-resolution rather than silently never firing. `supports_cost_log_tags` and
-`system_prompt_semantics` are declared for reasons of their own — see
+Capabilities are declared rather than probed, on the agent's `HarnessContract`.
+`contract.cooperative_stop` gates arming early-stop, so arming it on an agent that ignores
+`should_stop` is rejected at resolution rather than silently never firing.
+`contract.system_prompt_semantics` is declared for reasons of its own, and `cost_log_tags`
+is a base-constructor kwarg — see
 [agents.md](agents.md) § Why the constructors declare every kwarg and § The
 system_prompt_semantics marker.
 

CLAUDE.md

@@ -86,8 +86,10 @@ Each entry is a pointer. Full rationale: `.claude/notes/` (index: `.claude/notes
   Defense-in-depth, not a boundary — the known gaps are documented in the notes.
   Authoring reference: [Reference Solutions](docs/TASK_DEFINITION_GUIDE.md#reference-solutions).
 - **Harness run-limit parity**: a shared config field must mean the same thing on every
-  backend, or the divergence is documented. Table:
-  [Run-Limit Parity](docs/agents/HARNESS_PARITY.md). Caps are authored under
+  backend, or the divergence is documented. Every agent declares a `HarnessContract`; a
+  base field the harness marks unsupported is rejected at resolution. Table:
+  [Run-Limit Parity](docs/agents/HARNESS_PARITY.md) § Agent-field contract
+  (generated). Caps are authored under
   [Run Limits](docs/TASK_DEFINITION_GUIDE.md#run-limits).
 - **Execute vs. run**: `execute` is `run` with grading off — rows finalize as
   `NOT_GRADED` and leave both sides of every rate. Per-command behaviour:
@@ -162,6 +164,7 @@ make evalboard-verify   # the JS half: tsc --noEmit + vitest + next build
 make docs-indexes      # README/docs index tables from the mkdocs nav (CE028)
 make plugin-reference  # the plugin's criteria reference from the models (CE033)
 make pricing-mirror    # the evalboard's rate table from pricing.py (CE065)
+make parity-table      # the agent-field contract tables from the agent classes (CE069)
 
 make docs-budget       # per-file comment budget + docstring essay check (fails `make verify`)
 ```
@@ -215,6 +218,10 @@ A few rules constrain routine edits, so they are worth knowing before you start:
   metric, statistic or serializer pulled out of `reports*` is what put `turn_time_buckets`
   and the run.json serializer in a rendering module; they now live in `result_metrics.py`,
   `stats.py` and `run_record.py`.
+- **CE068** keeps `orchestration/`, `streaming/` and `timing.py` free of concrete agent
+  config classes and `AgentKind` members (except `UNKNOWN`); ask the registry instead.
+- **CE069** diffs the generated contract tables in `docs/agents/HARNESS_PARITY.md` against
+  the agent classes. Regenerate with `make parity-table`.
 
 **Docs index SSOT.** `nav:` plus `extra.docs_index` in `mkdocs.yml` are the single
 source of truth for `README.md`'s Documentation table, `docs/index.md`'s "Where to go
@@ -252,8 +259,9 @@ A live criterion also needs `ContractCase`s (CE036) and `make plugin-reference`.
 
 **A new agent**: agents register through the plugin SPI (entry-point group
 `coder_eval.plugins`) — there is no closed enum or dispatch to edit, and in-tree and
-third-party agents take the same path. A new agent must be named on every onboarding
-surface CE047 tracks, and its run-limit behaviour recorded in
+third-party agents take the same path. It declares a `HarnessContract` (registration
+fails without one) and imports from `coder_eval.spi`. A new agent must be named on every
+onboarding surface CE047 tracks, and its run-limit behaviour recorded in
 [Run-Limit Parity](docs/agents/HARNESS_PARITY.md).
 
 **Model pricing**: `register_pricing(YOUR_RATES)` from the same `register(registry)`

Makefile

@@ -1,4 +1,4 @@
-.PHONY: help install format check typecheck test test-live test-smoke verify verify-noextra evalboard-verify clean run lint docs-indexes plugin-reference pricing-mirror docs-budget docker-image docker-image-full coder-eval-runtime docker-images
+.PHONY: help install format check typecheck test test-live test-smoke verify verify-noextra evalboard-verify clean run lint docs-indexes plugin-reference pricing-mirror parity-table docs-budget docker-image docker-image-full coder-eval-runtime docker-images
 
 # Single source of the installed coder-eval version (used to tag the docker
 # images). Referenced lazily inside the docker recipes, so it doesn't run on
@@ -39,6 +39,9 @@ plugin-reference:  ## Regenerate the plugin's bundled criteria reference from th
 pricing-mirror:  ## Regenerate the evalboard's rate table from pricing.py (SSOT)
 	uv run python -m tests.lint.pricing_mirror
 
+parity-table:  ## Regenerate the agent-field contract table from the agent classes (SSOT)
+	uv run python -m tests.lint.harness_parity
+
 docs-budget:  ## Report the docstring/comment prose budget and check it against the baseline
 	uv run python -m tests.lint.prose_budget
 

README.md

@@ -253,7 +253,7 @@ The step's exit code is coder-eval's own: non-zero on any failed task.
 | [Antigravity (Gemini)](docs/agents/ANTIGRAVITY.md) | Running the Google Antigravity / Gemini agent |
 | [OpenCode](docs/agents/OPENCODE.md) | Running the OpenCode agent on open-weight models |
 | [Pi](docs/agents/PI.md) | Running the Pi agent on open-weight models |
-| [Run-Limit Parity](docs/agents/HARNESS_PARITY.md) | What each run_limits field means on every harness |
+| [Run-Limit Parity](docs/agents/HARNESS_PARITY.md) | What each run_limits and agent field means on every harness |
 | [A/B Experiments](docs/AB_EXPERIMENTS.md) | Compare models / tools / prompts across the same tasks |
 | [Bring Your Own Dataset](docs/DATASETS.md) | Fan a single task out over a dataset |
 | [Dialog Mode](docs/DIALOG_MODE.md) | Evaluate agents in multi-turn conversation via a simulated user |

docs/AB_EXPERIMENTS.md

@@ -113,6 +113,30 @@ it (the unification invariant). The per-field strategy is:
 Variants set the sandbox driver via `driver:` and add templates via
 `template_sources:` (top-level fields); they don't set a full `sandbox:` block.
 
+### Per-kind defaults with by_type
+
+Layers 1 and 2 may carry `by_type:` inside their `agent:` block. Each entry is a
+sub-layer that applies only when the resolved agent kind matches, and it sits
+directly above its own layer, so it stays **below the task**:
+
+```yaml
+defaults:
+  agent:
+    type: claude-code
+    by_type:
+      claude-code:
+        model: claude-sonnet-4-6
+        permission_mode: acceptEdits
+      pi:
+        model: openrouter/moonshotai/kimi-k3
+```
+
+The kind is the final `agent.type` across all five layers, so `--type pi` selects
+the `pi` entry and never inherits the `claude-code` one. An entry for a kind that is
+not installed is ignored. `by_type` is not allowed on a task or a variant (a task
+knows its kind; a variant sets its fields directly), and an entry must not set
+`type`. Lineage records such a value with `source_detail: by_type.<kind>`.
+
 ## What a Variant Can Override
 
 From `ExperimentVariant` (`coder_eval/models/experiment.py`):

docs/DIALOG_MODE.md

@@ -57,8 +57,9 @@ The mechanics:
   from its persona and goal — closer to a cold-start user.
 - Each exchange is one user message plus the agent's full response (the agent may make many tool
   calls inside a single exchange).
-- The simulator is a **tools-disabled Claude Code agent** with `allowed_tools: []`, an explicit
-  deny-list, and no plugins or settings sources. It is pure text-in / text-out, and it **cannot see
+- The simulator is a **tools-disabled Claude Code agent**: an explicit deny-list of every
+  built-in tool, and no plugins or settings sources. (Its `allowed_tools: []` restricts
+  nothing on Claude Code; the deny-list is the safeguard.) It is pure text-in / text-out, and it **cannot see
   the sandbox** — no files, no terminal, no agent reasoning. Only what the agent writes in the chat.
 - The simulator resolves its own `ApiRoute` independently of `checker_context.api_route` (that
   override is judge-only — see [Checker Context](TASK_DEFINITION_GUIDE.md#checker-context)) — same