feat: harden benchmark runs and stabilize TUI prompt bar

.gitignore

@@ -65,6 +65,7 @@ static/
 !.claude/hooks/
 !.claude/hooks/**
 .pythinker/
+.pi-subagents/
 .worktrees/
 reference-scan/
 blackbox/
@@ -85,4 +86,4 @@ htmlcov/
 .playwright/
 
 # Cursor debug-mode session logs (machine-local NDJSON)
-.cursor/debug-*.log
+.cursor/debug-*.log

CHANGELOG.md

@@ -15,6 +15,52 @@ GitHub Releases page; `0.8.0` is the new starting line.
 
 ## Unreleased
 
+- Show active subagent tool work in the pinned TUI status tail instead of
+  leaving long foreground agent runs on the generic composing spinner.
+- Keep the TUI prompt bar visible while an agent turn is starting so the
+  empty composer does not disappear during lazy-load frames.
+- Ported the running agent TUI toward Pi's stable diff-rendered scene model so streamed output keeps the input card visible without prompt jumps.
+- Added publishability-focused benchmark comparison planning for multi-model
+  runs, activity metrics, exportable reports, and safe online source discovery.
+- Added an experimental Focus TUI mode for active agent turns, keeping the composer pinned, hiding file activity by default, and rendering live output without terminal scrollback jumps.
+- Stream file write/edit activity in a compact live shelf so changed files update in place during agent runs instead of adding noisy terminal rows.
+- Hardened `/benchmark` local fixture runs: task `max_steps` now caps the
+  underlying agent turn, and `/benchmark:swe` requires `--trusted-dataset true`
+  because trusted local fixture datasets execute verification commands.
+- Hardened benchmark and active-skill security edges: SWE verification commands
+  are shape-validated before execution, benchmark discovery requires explicit
+  network opt-in, benchmark run IDs avoid clock collisions, runtime overrides
+  restore after setup failures, and active-skill deactivation persistence
+  failures are surfaced instead of swallowed.
+- Strengthened the bundled `pythinker-core` benchmark suite with edge-case
+  fixtures for atomic rollback, iterable de-duplication, explicit falsey
+  metadata values, and safe path joins across absolute, sibling-prefix, parent,
+  and symlink escapes.
+
+- Keep the terminal input composer pinned to the bottom during agent runs with a fullscreen prompt mode to reduce TUI flicker.
+
+- Explicitly invoked skills now remain active across later turns through a
+  compact reminder, and can be cleared with a named stop request or "normal mode".
+
+- **No more ghost/duplicate input prompt while the agent works.** After
+  submitting a prompt, the editable input row is no longer fossilized above the
+  stream as a second, ghostly prompt. The top border stays visible while the
+  pre-attach race frame still collapses before the running-prompt delegate
+  exists, preventing prompt chrome from fossilizing above the spinner. Once the
+  running frame owns the prompt, the `❯` marker stays visible while only the
+  editable buffer is hidden until the first scrollback commit, avoiding the
+  collapsed one-line card under the lazy-load spinner; the full editable row
+  remains below the live stream so you can still see where to steer.
+
+- Added native `/benchmark` slash command for deterministic local Pythinker
+  model evaluation with bundled smoke tasks, replayable artifacts, and branded
+  markdown reports.
+- Expanded `/benchmark start` to use a richer default core suite, isolate file
+  edits through the active toolset workspace override, and exclude generated
+  verification caches from changed-file reports.
+- Added `/benchmark:swe` for native SWE-style JSONL benchmark tasks that run
+  through Pythinker's existing model, tool, verification, and artifact path.
+
 ## 0.56.0 (2026-07-02)
 
 - **Windows shell UI recovers from mid-session console blanking.** The TUI's

README.md

@@ -126,6 +126,22 @@ Optional web frontend and visualization frontend ship alongside the CLI for rich
 
 Swap providers and models per-session: `--model openai/gpt-5.5`, hosted Pythinker models, or your own keys.
 
+</td>
+</tr>
+<tr>
+<td width="50%" valign="top">
+
+### 📊 Local Benchmarks
+
+Run `/benchmark` to execute deterministic local coding tasks through the active Pythinker session, with replayable artifacts and verification reports.
+
+</td>
+<td width="50%" valign="top">
+
+### 🧪 SWE-Style Fixtures
+
+Run trusted local JSONL fixtures with `/benchmark:swe --dataset <path> --trusted-dataset true` when you want SWE-style task inputs without a hosted evaluator.
+
 </td>
 </tr>
 </table>
@@ -618,6 +634,7 @@ Pythinker is a small, extensible runtime — not a monolith. Build on it.
 | 🌊 **Flows** | `/flow:<name>` executes bundled prompt flows | bundled & user-defined |
 | 🪝 **Hooks** | Observe or block tool execution; integrate policy or automation | hook events API |
 | 🧩 **Plugins** | Installable extension packages | `pythinker plugin` |
+| 📊 **Benchmarks** | Deterministic local coding tasks with verification reports | `/benchmark`, `src/pythinker_code/benchmark/` |
 
 ---
 

docs/.vitepress/config.ts

@@ -83,6 +83,7 @@ export default withMermaid(defineConfig({
                 { text: 'pythinker term Subcommand', link: '/en/reference/pythinker-term' },
                 { text: 'pythinker dashboard Subcommand', link: '/en/reference/pythinker-dashboard' },
                 { text: 'pythinker web Subcommand', link: '/en/reference/pythinker-web' },
+                { text: 'Pythinker Benchmark', link: '/en/reference/pythinker-benchmark' },
                 { text: 'Slash Commands', link: '/en/reference/slash-commands' },
                 { text: 'Keyboard Shortcuts', link: '/en/reference/keyboard' },
               ],

docs/AGENTS.md

@@ -9,7 +9,7 @@ This repository uses VitePress for the documentation site. Most pages now contai
   - Guides: getting-started, use-cases, interaction, sessions, ides, integrations
   - Customization: mcp, plugins, hooks, skills, agents, print-mode, wire-mode
   - Configuration: config-files, providers, overrides, env-vars, data-locations
-  - Reference: pythinker-command, pythinker-info, pythinker-acp, pythinker-mcp, pythinker-term, pythinker-dashboard, pythinker-web, slash-commands, keyboard
+  - Reference: pythinker-command, pythinker-info, pythinker-acp, pythinker-mcp, pythinker-term, pythinker-dashboard, pythinker-web, pythinker-benchmark, slash-commands, keyboard
   - FAQ: faq
   - Release notes: changelog, breaking-changes
 - Navigation and sidebar are defined in `docs/.vitepress/config.ts`. Any new or renamed page must be wired there.

docs/en/customization/architecture.md

@@ -166,6 +166,38 @@ Provider modules in `auth/`: `openai`, `anthropic_direct`, `opencode_go`, `minim
 follow `<platform_id>/<model_id>`. Provider-aware code derives the provider from the active
 model; `/usage` defaults to the active provider, with `/usage all` as the explicit aggregate.
 
+## Benchmark runner
+
+Native benchmark execution lives under `src/pythinker_code/benchmark/` and is surfaced through
+the slash-command registry in `src/pythinker_code/soul/slash.py`. It is a local-fixture harness:
+tasks materialize files into a per-run workspace, run through the same `PythinkerSoul.turn`
+path as a normal session, then execute a verification command and persist artifacts.
+
+| Path | Purpose | Key entry points and interfaces |
+| --- | --- | --- |
+| `src/pythinker_code/benchmark/commands.py` | Slash-command parser and orchestrator for `start`, `estimate`, `list`, `show`, `report`, `export`, `compare`, `discover`, and `swe`. | `dispatch_benchmark`, `BenchmarkArgs`, `benchmark_usage` |
+| `src/pythinker_code/benchmark/compare.py` and `export.py` | Publishability warnings and report-row export formatting for model comparisons. | `readiness_warnings`, `render_export` |
+| `src/pythinker_code/benchmark/discovery.py` | Allowlisted online source discovery and provisional quiz-fixture JSONL conversion. | `discover_benchmark_sources`, `quiz_fixture_from_discovery` |
+| `src/pythinker_code/benchmark/runner.py` | Per-task execution: workspace materialization, work-dir override, temporary task `max_steps` limit, timeout handling, verification, and artifact finalization. | `run_task`, `BenchmarkResult`, `VerificationResult` |
+| `src/pythinker_code/benchmark/tasks.py` | Bundled task schema and workspace materialization. | `BenchmarkTask`, `load_task`, `materialize_workspace` |
+| `src/pythinker_code/benchmark/suites.py` | Bundled suite loading and ordering. | `load_suite`, `list_suite_names` |
+| `src/pythinker_code/benchmark/swe.py` | Trusted local SWE-style JSONL fixture loading. | `load_swe_instances`, `swe_instance_to_task` |
+| `src/pythinker_code/benchmark/records.py` and `report.py` | Per-run artifact writing and report rendering. | `BenchmarkRecorder`, `render_run_report`, `render_show` |
+
+The bundled suite files live in `src/pythinker_code/benchmark/bundled/suites/`; bundled task
+definitions live in `src/pythinker_code/benchmark/bundled/tasks/`. The `pythinker-core` suite
+targets common agent failure modes: transactional rollback, ordered de-duplication, explicit
+`None` versus falsey metadata, and safe path joins. The runner uses the active model provider
+from session config; it does not create a separate provider stack.
+
+`/benchmark:swe` is intentionally labeled SWE-style local fixture support, not full SWE-bench
+Docker evaluation. It accepts local JSONL records, rejects unsafe workspace paths, and requires
+`--trusted-dataset true` before running dataset-provided verification commands.
+
+`/benchmark discover` only writes provisional review manifests when `--output` ends in
+`.jsonl`; those records are untrusted quiz fixtures with empty workspaces, not runnable
+SWE-style local fixtures.
+
 ## Wire and UI frontends
 
 | Path | Purpose | Key entry points and interfaces |

docs/en/reference/pythinker-benchmark.md

@@ -0,0 +1,194 @@
+# Pythinker Benchmark
+
+Pythinker Benchmark is the native local-fixture harness for comparing how a configured Pythinker model handles small, deterministic coding tasks. It runs inside the current Pythinker session, uses the active toolset and approval runtime, materializes each task into an isolated workspace, executes the agent turn, then runs the task's verification command and writes replayable artifacts.
+
+It is designed for repeatable local checks, not hosted leaderboard scoring. SWE-style input is supported as trusted local fixtures; it is not a full SWE-bench Docker runner.
+
+## Commands
+
+Run the default core suite:
+
+```sh
+/benchmark start
+```
+
+Run a single task or named suite:
+
+```sh
+/benchmark start --task core-safe-path-join
+/benchmark start --suite pythinker-smoke
+```
+
+Estimate a run without making model calls:
+
+```sh
+/benchmark estimate --suite pythinker-core
+```
+
+Inspect available tasks and saved runs:
+
+```sh
+/benchmark list
+/benchmark show <run-id>
+/benchmark report --suite pythinker-core
+```
+
+Compare configured models and export report data:
+
+```sh
+/benchmark compare --models model-a,model-b --suite pythinker-core --repeat 3
+/benchmark export --suite pythinker-core --format csv
+```
+
+Discover candidate tasks from an allowlisted online source:
+
+```sh
+/benchmark discover --source terminal-bench --difficulty hard --limit 5 --output ./candidate-tasks.jsonl
+```
+
+Namespaced aliases are available for interactive completion: `/benchmark:start`, `/benchmark:all`, `/benchmark:estimate`, `/benchmark:list`, `/benchmark:show`, `/benchmark:report`, `/benchmark:compare`, and `/benchmark:swe`. There is no `/benchmark:export` or `/benchmark:discover` alias.
+
+The supported flags are:
+
+| Flag | Applies to | Behavior |
+| --- | --- | --- |
+| `--model <model-key>` | `start`, `estimate`, `swe` | Uses a configured model instead of the active/default model. |
+| `--models <model-a,model-b>` | `compare` | Runs each selected task for at least two distinct configured models. |
+| `--task <task-id>` | `start`, `estimate`, `compare` | Runs, estimates, or compares one bundled task. Mutually exclusive with `--suite`. |
+| `--suite <suite-name>` | `start`, `estimate`, `report`, `export`, `compare` | Selects a bundled suite or filters report/export output. |
+| `--repeat <n>` | `start`, `estimate`, `compare`, `swe` | Runs or estimates each selected task multiple times. |
+| `--timeout-seconds <n>` | `start`, `compare`, `swe` | Overrides the task timeout for the agent turn. |
+| `--format json\|csv` | `export` | Selects the export format. |
+| `--output <path>` | `start`, `compare`, `show`, `report`, `export`, `swe`; `.jsonl` only for `discover` | Uses a custom benchmark artifact root for run/report/export commands. For `discover`, it only writes a provisional manifest when the path suffix is `.jsonl`. |
+| `--dataset <path.jsonl>` | `swe` | Loads SWE-style local fixture records from a JSONL file. |
+| `--instance <instance-id>` | `swe` | Runs only one instance from the dataset. |
+| `--trusted-dataset true` | `swe` | Required acknowledgement before dataset verification commands can run. |
+| `--source <allowlisted>` | `discover` | Selects an allowlisted benchmark source such as `terminal-bench`. |
+| `--difficulty <difficulty>` | `discover` | Filters discovered candidates by difficulty. Defaults to `hard`. |
+| `--limit <n>` | `discover` | Limits discovered candidates. Defaults to `5`. |
+
+`--max-concurrency` is parsed but must remain `1` in the current implementation. `--judges` is parsed but only `off` is supported.
+
+## Architecture
+
+The benchmark integration has four layers:
+
+1. Slash commands in `src/pythinker_code/soul/slash.py` register `/benchmark` and the namespaced aliases.
+2. `src/pythinker_code/benchmark/commands.py` parses arguments, resolves the active model, expands tasks or suites, and dispatches each run.
+3. `src/pythinker_code/benchmark/runner.py` prepares the workspace, temporarily applies the task's `max_steps` to the soul loop, overrides the runtime work directory, calls `PythinkerSoul.turn`, runs verification, and restores the previous runtime state in a `finally` block.
+4. `src/pythinker_code/benchmark/records.py` and `src/pythinker_code/benchmark/report.py` persist run metadata, traces, summaries, context and Wire slices, and Markdown reports.
+
+Task and suite definitions are regular bundled JSON files under `src/pythinker_code/benchmark/bundled/`. `src/pythinker_code/benchmark/tasks.py` validates bundled task shape and rejects unsafe workspace paths before files are materialized.
+
+## Bundled suites
+
+`pythinker-core` is the default suite. It covers deterministic local coding tasks that expose common agent failure modes:
+
+- `core-atomic-transfer`: transactional rollback, missing accounts, insufficient funds, and non-positive transfer amounts.
+- `core-dedup-order`: ordered de-duplication for lists and generators, including falsey values.
+- `core-explicit-none-metadata`: explicit `None` handling without losing valid falsey metadata values.
+- `core-safe-path-join`: path traversal defense, absolute-path rejection, sibling-prefix escapes, and symlink escapes.
+
+`pythinker-smoke` contains smaller tasks for validating the runner itself:
+
+- `smoke-edit-readme`
+- `smoke-fix-python-test`
+- `smoke-add-small-function`
+
+## Task schema
+
+A bundled task JSON object contains:
+
+```json
+{
+  "id": "core-safe-path-join",
+  "title": "Safe Path Join",
+  "description": "Reject path traversal while allowing paths inside the root.",
+  "prompt": "Fix paths.safe_join ...",
+  "workspace": {
+    "files": {
+      "paths.py": "...",
+      "test_paths.py": "..."
+    }
+  },
+  "verification": {
+    "type": "command",
+    "command": "python -m pytest test_paths.py -q"
+  },
+  "limits": {
+    "timeout_seconds": 120,
+    "max_steps": 60
+  },
+  "tags": ["core", "security"]
+}
+```
+
+Workspace paths must be relative, non-empty, and must not contain `..` path segments. Verification type is currently `command`.
+
+## Publishable comparisons
+
+Use `/benchmark compare` when comparing configured models:
+
+```sh
+/benchmark compare --models model-a,model-b --suite pythinker-core --repeat 3
+```
+
+Export the saved report rows when you need machine-readable results:
+
+```sh
+/benchmark export --suite pythinker-core --format csv
+```
+
+Reports include publishability warnings. Treat warnings as blockers for public claims, not as lint. Local fixture runs are useful for regression and internal comparison, but they are not SWE-bench Docker evaluations.
+
+## Online discovery and quiz fixtures
+
+`/benchmark discover` fetches metadata from allowlisted benchmark sources and writes provisional manifests. It does not execute source-provided commands and does not make discovered tasks trusted:
+
+```sh
+/benchmark discover --source terminal-bench --difficulty hard --limit 5 --output ./candidate-tasks.jsonl
+```
+
+The `--output` flag writes a manifest only when the path suffix is `.jsonl`. These JSONL records preserve the source URL and are marked `trusted: false`. Online quiz fixture records use deterministic review metadata:
+
+```json
+{
+  "verification": {
+    "type": "answer_contains",
+    "expected_substrings": ["terminal-bench", "hard"]
+  },
+  "trusted": false,
+  "workspace": {
+    "files": {}
+  }
+}
+```
+
+These manifests are for dataset review and offline conversion first. They are not runnable through `/benchmark:swe`, and `answer_contains` is not executed by the benchmark runner. Convert reviewed tasks into trusted local fixtures with workspace files and a local verification command before running them.
+
+## SWE-style local fixtures
+
+`/benchmark:swe` loads newline-delimited JSON records with local workspace files and a verification command. The command is executed on the local machine after the agent turn, so the slash command refuses to run unless `--trusted-dataset true` is present:
+
+```sh
+/benchmark:swe --dataset ./cases.jsonl --trusted-dataset true
+```
+
+Each record must include `instance_id`, `repo`, `base_commit`, `problem_statement`, `workspace.files`, and `verification.command`. Optional `FAIL_TO_PASS`, `PASS_TO_PASS`, and `limits` fields are folded into the generated local fixture task.
+
+## Artifacts
+
+By default, benchmark runs are written under the Pythinker share directory in `benchmarks/<run-id>/`. A custom root can be supplied with `--output <path>`.
+
+Each run directory contains:
+
+| File or directory | Contents |
+| --- | --- |
+| `run.json` | Run metadata: command, model key, provider key, task id, suite name, repeat index, timestamps, and final status. |
+| `summary.json` | Runtime summary: duration, step count, tool calls, changed files, token counts, verification status, and exit reason. |
+| `report.md` | Human-readable report for the run. |
+| `trace.jsonl` | Benchmark event trace, including workspace preparation, model message, and verification result. |
+| `workspace/` | The materialized local task workspace after the run. |
+| `context.jsonl` and `wire.jsonl` | Slices copied from the active session for replay and debugging. |
+
+Generated verification caches such as `__pycache__`, `.pytest_cache`, `.ruff_cache`, and `.mypy_cache` are excluded from changed-file summaries.