Skip to content

sandbox.python.env_packages / node.env_packages are never installed under driver: docker #188

Description

@akshaylive

Summary

sandbox.python.env_packages (and the Node equivalent, sandbox.node.env_packages) are only ever installed by the host-side Sandbox.provision() path (_install_packages / _install_node_packages in src/coder_eval/sandbox.py), which backs driver: tempdir.

src/coder_eval/isolation/docker_runner.py has no equivalent step at all — no pip/uv pip install, no npm install, nothing. So any task with driver: docker plus sandbox.python.env_packages (or sandbox.node.env_packages) silently never gets those packages installed inside the container. Neither the docs (docs/TASK_DEFINITION_GUIDE.md) nor the field description on PythonEnvConfig/NodeEnvConfig (src/coder_eval/models/sandbox.py) mention this as a driver: docker limitation, so this reads as a real functionality gap rather than a documented restriction.

How it surfaced

The Harbor E2E template_sources scenario (tests/harbor_e2e/fixtures/template_sources.yaml, driver: docker + sandbox.python.env_packages: [pytest]) failed in CI (#187) with:

Command: python -m pytest tests/
Exit code: 1 (expected: 0)
Stderr:
/usr/local/bin/python: No module named pytest

The agent's own recorded trajectory shows it also tried python -m pytest tests/test_main.py -v and then fell back to manually verifying its implementation with a bare python -c check — consistent with pytest never having been installed in its own sandbox either.

Suggested fix

Teach DockerRunner to provision sandbox.python.env_packages / sandbox.node.env_packages inside the container (an exec step analogous to Sandbox._install_packages/_install_node_packages) before the agent starts, or explicitly document the gap if it's intentionally out of scope for driver: docker today.

Scope note

Confirmed via code inspection that this predates #186 (c9b96a80) — it is not a regression from that commit, just the first thing to exercise this path in CI. Filed separately per triage of https://github.com/UiPath/coder_eval/actions/runs/35242305732 rather than folding into PR #187.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions