No eval for multi-turn behavior: handles, chaining, and recovery

[rajeeja]

Problem

This server ships a fair amount of multi-step machinery — create_session, dataset_handle, result_handle, run_workflow, resume_workflow, get_status — and almost none of it is measured with a model in the loop.

tests/test_stateful_tools.py and friends verify the tools work when called correctly from Python. What we do not measure is whether a model can actually:

• mint a handle and carry it across several calls without dropping or inventing it
• chain one tool's output into the next tool's input
• notice a failure mid-sequence and recover, rather than continuing on a bad result
• resume a workflow after an interruption

In a 480-run evaluation of this server the median run made one or two tool calls. Multi-step behavior was effectively untested, and that is the capability autonomous use actually depends on.

Suggested direction

An eval under evals/, following the existing pattern there (fixed inputs, numeric score, committed runner and results):

• a small set of tasks that cannot be completed in a single call, so chaining is forced
• score handle discipline: was the handle reused correctly, invented, or dropped
• score recovery: inject a deliberate failure partway through and record whether the model detects it and retries versus proceeding on bad state
• report per-model, since the earlier evaluation showed pooled numbers hide models at ceiling and models at floor

Notes

• This is measurement, not a fix. It will likely surface issues rather than close them, which is the point.
• It also gives Warnings inform but never block #86 (blocking preconditions) something to be validated against: a refusal is only useful if the caller does something sensible afterwards.

[rajeeja]

Fixed in #97, now merged to main.