Problem
The Terminal-Bench adapter builds its Python and tb paths with join(benchRoot, terminalBenchVenvDir(), ...). If TERMINAL_BENCH_VENV is absolute, path.join produces an invalid path under the package root instead of respecting the absolute venv.
Observed from agent-lab R309 live pilot:
spawn /home/drew/code/agent-lab-k-law-r309/node_modules/.pnpm/@tangle-network+agent-bench@file+..+agent-runtime+bench.../node_modules/@tangle-network/agent-bench/home/drew/code/agent-runtime/bench/.venv-terminal-bench/bin/python ENOENT
The same venv works when passed as a package-relative path:
TERMINAL_BENCH_VENV=../../../../../../../agent-runtime/bench/.venv-terminal-bench
Expected
TERMINAL_BENCH_VENV=/absolute/path/to/.venv-terminal-bench should resolve to /absolute/path/to/.venv-terminal-bench/bin/python and /absolute/path/to/.venv-terminal-bench/bin/tb.
Suggested fix
In bench/src/benchmarks/terminal-bench.ts, resolve venv paths with path.isAbsolute before joining with benchRoot, similar to:
const terminalBenchVenvPath = () => {
const dir = process.env.TERMINAL_BENCH_VENV ?? ".venv-terminal-bench"
return isAbsolute(dir) ? dir : join(benchRoot, dir)
}
Then build python and tb paths from that resolved venv path.
Why it matters
agent-lab needs to run published @tangle-network/agent-bench from a worktree while reusing the real Terminal-Bench venv in ~/code/agent-runtime/bench. The current path handling forces fragile package-relative paths in live benchmark commands.
Problem
The Terminal-Bench adapter builds its Python and
tbpaths withjoin(benchRoot, terminalBenchVenvDir(), ...). IfTERMINAL_BENCH_VENVis absolute,path.joinproduces an invalid path under the package root instead of respecting the absolute venv.Observed from agent-lab R309 live pilot:
The same venv works when passed as a package-relative path:
Expected
TERMINAL_BENCH_VENV=/absolute/path/to/.venv-terminal-benchshould resolve to/absolute/path/to/.venv-terminal-bench/bin/pythonand/absolute/path/to/.venv-terminal-bench/bin/tb.Suggested fix
In
bench/src/benchmarks/terminal-bench.ts, resolve venv paths withpath.isAbsolutebefore joining withbenchRoot, similar to:Then build
pythonandtbpaths from that resolved venv path.Why it matters
agent-lab needs to run published
@tangle-network/agent-benchfrom a worktree while reusing the real Terminal-Bench venv in~/code/agent-runtime/bench. The current path handling forces fragile package-relative paths in live benchmark commands.