Skip to content

fix(agent-bench): support absolute TERMINAL_BENCH_VENV paths #475

Description

@drewstone

Problem

The Terminal-Bench adapter builds its Python and tb paths with join(benchRoot, terminalBenchVenvDir(), ...). If TERMINAL_BENCH_VENV is absolute, path.join produces an invalid path under the package root instead of respecting the absolute venv.

Observed from agent-lab R309 live pilot:

spawn /home/drew/code/agent-lab-k-law-r309/node_modules/.pnpm/@tangle-network+agent-bench@file+..+agent-runtime+bench.../node_modules/@tangle-network/agent-bench/home/drew/code/agent-runtime/bench/.venv-terminal-bench/bin/python ENOENT

The same venv works when passed as a package-relative path:

TERMINAL_BENCH_VENV=../../../../../../../agent-runtime/bench/.venv-terminal-bench

Expected

TERMINAL_BENCH_VENV=/absolute/path/to/.venv-terminal-bench should resolve to /absolute/path/to/.venv-terminal-bench/bin/python and /absolute/path/to/.venv-terminal-bench/bin/tb.

Suggested fix

In bench/src/benchmarks/terminal-bench.ts, resolve venv paths with path.isAbsolute before joining with benchRoot, similar to:

const terminalBenchVenvPath = () => {
  const dir = process.env.TERMINAL_BENCH_VENV ?? ".venv-terminal-bench"
  return isAbsolute(dir) ? dir : join(benchRoot, dir)
}

Then build python and tb paths from that resolved venv path.

Why it matters

agent-lab needs to run published @tangle-network/agent-bench from a worktree while reusing the real Terminal-Bench venv in ~/code/agent-runtime/bench. The current path handling forces fragile package-relative paths in live benchmark commands.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions