fix(docker): pin uv in the agent image — 0.12.11+ copies cross-fs ~40x slower

[tmatup]

Summary

Short version: the agent image installed uv unpinned, and uv 0.12.11+ makes the cross-filesystem copy (cache in the image, venv on the bind-mounted workspace) about 40× slower. Coded-agent tasks have paid an extra ~2 min of wall clock since the 0.12.1 image (2026-09-12). This PR pins uv to 0.12.10, the last good release.

• docker/Dockerfile: ARG UV_VERSION=0.12.10 + https://astral.sh/uv/${UV_VERSION}/install.sh, with the reason in a comment. Same pattern as the Claude Code and Pi pins in that file.
• tests/test_image_from_dockerfiles.py: test_uv_pinned_in_agent_image fails on an unversioned installer URL or a missing or non-exact pin.

Evidence

CI, skill-flow-coded-agent (UiPath/skills nightlies + adhoc v1/v2 runs):

dates	uv in image (environment_info)	Installed 146 packages in …
up to 09-11	0.12.9	~1–10s
09-14 onward	0.12.12 → 0.12.19	23–132s

The package set is unchanged (145–146). Resolve and download take under a second (Prepared 118 packages in 815ms). All the time is in the link step after warning: Failed to hardlink files; falling back to full copy. In adhoc-2026-09-30_19-54-19, one scaffold command spent Installed 146 packages in 1m 05s plus Installed 31 packages in 54.08s. Codex's exec timeout killed four such installs (exit 130), and the agents' retries showed up as extra uip/uv commands.

Local repro (Docker, bind mount, warm cache, fresh venv, uv pip install uipath-langchain):

uv	time
0.12.9	0.28s
0.12.10	0.31s
0.12.11	10.62s
0.12.12	10.64s
0.12.19	10.42s
0.12.21	14.2s

On the same filesystem it is 0.2s on every version. strace -c shows the same syscall counts, but openat time goes from 3.3s to 11.5s. The 0.12.11 release notes touch this path (astral-sh/uv#21468, #21478).

This image vs the published one (same bind mount):

• ghcr.io/uipath/coder-eval-agent:0.12.8 (uv 0.12.19): 10.60s, 10.68s
• this branch's image (uv 0.12.10): 0.31s, 0.21s

Why a pin and not UV_LINK_MODE=symlink

Symlink mode is fast (0.05s), but it leaves .venv pointing into the container's cache. Detached grading runs in a second container, and the artifact upload reads the workspace after the container exits. Both would see dangling links. The pin restores the behavior every run had before 09-12. Bump UV_VERSION deliberately, after timing an install into a bind-mounted dir.

Scope notes

• docker/Dockerfile.runtime also uses the unversioned installer. I left it alone: there, uv lives only in the builder stage (/usr/local/bin), and only /opt/coder-eval is copied into the final image. That uv never reaches a task image.
• The pin removes the step (CI went from about 4s to 23–132s per install on 09-14). The spread inside the slow period (41s → 131s across September) is host-side and not diagnosed here. Don't expect every run to come back at exactly 4s.
• To take effect in the skills evals, tests/.coder-eval-version in UiPath/skills (currently 0.12.8) has to move to the release this PR cuts. Both the nightly (daily.sh) and run-coder-eval.yml resolve the agent image tag from that file.

Test plan

• pytest tests/test_image_from_dockerfiles.py: 30 passed
• New test fails against the old docker/Dockerfile
• ruff check / ruff format --check
• docker build -f docker/Dockerfile → uv --version = 0.12.10; timed install above

🤖 Generated with Claude Code
Co-Authored-By: Claude

🤖 Generated with Claude Code

https://claude.ai/code/session_015MzgU89VtXESbzJfAAu9ay

[github-actions]

Claude finished @tmatup's task in 1m 15s —— View job

---

Code Review in Progress

• Read review guidelines (.github/code_review.md)
• Review full diff and context
• Check cross-file consistency
• Analyze what's missing
• Post review feedback

[uipreliga]

Review: coder_eval — pr:209 (2 files) axis:1,2,3,4,5,6,7,8

Scope: pr:209 (2 files) axis:1,2,3,4,5,6,7,8 · branch fix/pin-uv-agent-image (PR #209 by @tmatup → main, OPEN) · 4e0aff9 · 2026-10-01T04:42Z · workflow variant

Change class: simple — pins the uv version in the agent Dockerfile via an ARG and adds a static text-match test; no control flow, schema, or API change

The repo is healthy: seven of eight axes score 10/10 (Code Quality & Style is weakest at 9.4) and no confirmed finding can change a task's score or final_status for identical agent output, so the real risks are only a uv install step that fails silently and gives a misleading build error, plus small duplication and history-comment drift in the uv-pin change; the bottom line is to merge after the short fixes below.

Summary

Axis	Score	🔴	🟠	🟡	🔵	Top Issue
1. Code Quality & Style	9.4 / 10	0	0	1	1	Third copy of the inline ARG <NAME>= pin-parsing loop; the sibling _pin helper was not lifted and reused
2. Type Safety	10 / 10	0	0	0	0	—
3. Test Health	10 / 10	0	0	0	0	—
4. Security	10 / 10	0	0	0	0	—
5. Architecture & Design	10 / 10	0	0	0	0	—
6. Error Handling & Resilience	9.9 / 10	0	0	0	1	A failed versioned uv download makes the install layer exit 0 and the build later fails with "uv: not found"
7. API Surface & Maintainability	10 / 10	0	0	0	0	—
8. Evaluation Harness Quality	10 / 10	0	0	0	0	—

Overall Score: 9.9 / 10 · Weakest Axis: Code Quality & Style at 9.4 / 10
Totals: 🔴 0 · 🟠 0 · 🟡 1 · 🔵 2 across 8 axes.

Blockers

None.

Non-blocking, but please consider before merge

1. [Axis 1] Third copy of the inline ARG <NAME>= pin-parsing loop; the sibling _pin helper was not lifted and reused (tests/test_image_from_dockerfiles.py:495-502) — The new test copies the PI_VERSION loop line for line. Lines 495-502 are df_text = (Path(__file__).resolve().parents[1] / "docker" / "Dockerfile").read_text(encoding="utf-8") / pin: str | None = None / for ln in df_text.splitlines(): / m = re.match(r"\s*ARG UV_VERSION=(\S+)", ln) / if m: pin = m.group(1); break. The same loop is at 523-530 (ARG PI_VERSION=), and a nested _pin helper does the same work at 473-478 (ARG CLAUDE_CODE_VERSION=). That makes three sites of the same pin extraction, and two sites of the framework-Dockerfile path expression (495, 523). This matches the Axis 1 anchor 'duplication across 2–3 sites' (Medium). Move _pin to module level as _arg_pin(df_text: str, name: str) -> str | None, next to the existing _runtime_dockerfile() helper at line 416. Add a _framework_dockerfile_text() helper too. Then each pin test is about 3 lines, and the next pinned tool does not add a fourth copy.

Nits

1. [Axis 1] The uv pin comment in the Dockerfile holds history and benchmark notes that the test docstring repeats (docker/Dockerfile:45-54) — The new comment is a 10-line run. It records history that belongs in git or .claude/notes/, not next to the code: install: 0.3s on 0.12.10, 10-14s on 0.12.11/.12/.19/.21; 60-130s on the CI / host), and the unpinned install picked it up silently in the 0.12.1 image / (2026-09-12, uv 0.12.12). The CLAUDE.md principle says 'what it used to be belongs in git'. Also, '0.12.1 image' means coder_eval release v0.12.1 (CHANGELOG.md:898), but it sits between uv versions 0.12.10/0.12.11/0.12.12, so a reader can easily take it as a uv version. Reduce the comment to the contract: why uv is pinned (cross-filesystem copy into the bind-mounted workspace is about 40x slower from 0.12.11), and that a bump requires timing an install into a bind-mounted dir. Move the benchmark table and the incident date to a .claude/notes/ entry or the commit message. If a version reference must stay, write 'coder_eval v0.12.1'. The test docstring at tests/test_image_from_dockerfiles.py:489-494 has the same history ('moved the image from uv 0.12.9 to 0.12.12 on a rebuild'). Reduce it to the contract stated in its last sentence.
2. [Axis 6] A failed versioned uv download makes the install layer exit 0 and the build later fails with "uv: not found" (docker/Dockerfile:56) — Line 56 RUN curl -LsSf https://astral.sh/uv/${UV_VERSION}/install.sh | env UV_INSTALL_DIR=/usr/local/bin sh runs under Docker's default /bin/sh -c, which has no pipefail. The pipeline's exit status is that of sh. If curl fails, -f exits 22 and writes nothing, sh reads an empty script, and the step exits 0. The pin adds a new way for curl to fail: a mistyped or withdrawn version on a deliberate bump returns 404. A transient astral.sh 5xx or network blip causes the same result. The step is then reported as successful (and BuildKit can cache it as successful). The build only fails two steps later at line 107 (uv export --frozen ... | uv pip install ...) with "uv: not found", which hides the real cause: the pinned version could not be fetched. Make the failure happen at the step that caused it. Add SHELL ["/bin/bash", "-o", "pipefail", "-c"] before this RUN, or download the script first (curl -LsSf -o /tmp/uv-install.sh https://astral.sh/uv/${UV_VERSION}/install.sh && env UV_INSTALL_DIR=/usr/local/bin sh /tmp/uv-install.sh). Then append && uv --version | grep -F "${UV_VERSION}" so the layer also confirms that the pinned version is the one installed. This bug existed before the PR, with the unversioned URL, but this PR edits the line and adds the 404 case. Severity is Low because the build still fails, only with the wrong cause and at the wrong step. n/a

What's Missing

Nightly pipeline:

• 🟡 The PR gives the CI-host speedup (60-130s back to about 4s per install), but it does not say what happens to the nightly. (a) The fix reaches the nightly only after release.yml publishes a new coder-eval-agent: tag and the nightly moves to that tag. Until then, nightlies on the published 0.12.8/0.12.9 images (uv 0.12.19) stay slow. (b) Runs from 09-14 until the new image are a different cohort. The commit message says Codex's exec timeout killed uv installs (exit 130) and the retries added commands. So pass rate, turn count, command count and wall clock on coded-agent/function tasks all move at this boundary, and not only duration. State the rollout path, and add an annotation to the evalboard trend at the image switch, so that no one reads the step change as an agent or model change. (trigger: docker/Dockerfile)

Parallel paths:

• 🟡 The run records do not capture the uv version. The CLAUDE_CODE_VERSION pin is visible in every run as environment_info.claude_code_cli (src/coder_eval/utils.py:480), and docs/agents/CLAUDE_CODE.md tells consumers to segment on it. The new UV_VERSION pin has no matching field. This is why the 0.12.1 image's silent move to uv 0.12.12 went unattributed for about two weeks of nightlies. Record uv --version in environment_info next to claude_code_cli, so that the next uv bump can be seen in run.json and in evalboard cohorts. (trigger: docker/Dockerfile)
• 🔵 After this PR, the uv that writes uv.lock and the uv that reads it are different versions. CI (pip install uv, pr-checks.yml:88-91, setup-uv:265) and release.yml's prerelease uv lock (line ~194) use the latest uv. The image's uv export --frozen (docker/Dockerfile:107) now runs on the frozen 0.12.10. If a later uv writes a lock revision that 0.12.10 cannot read, the image build fails on the release path. No signal tells anyone to bump the pin (dependabot does not track Dockerfile ARGs, and there is no tracking issue for a fixed upstream uv). Pin the CI/release uv to the same version, or add a guard and a tracking item. (trigger: docker/Dockerfile)
• 🔵 docker/Dockerfile.runtime:38 still uses the unversioned installer. The PR body explains that the speed issue does not apply, because uv is only in the builder stage. But that uv also runs uv python install ${PYTHON_VERSION} with PYTHON_VERSION=3.13, and each uv release bundles its own python-build-standalone list. So the runtime kit's CPython patch version still changes when uv changes on a rebuild. Also, the comment on line 36 ('same tool the main image and host sandbox use') now hides a version difference. Pin it to the same UV_VERSION, or record in the Dockerfile comment why patch drift is acceptable. (trigger: docker/Dockerfile)
• 🔵 Two other paths can hit the same cross-filesystem slowdown, and nothing tells users. (1) Custom and BYO task images (the docs/DOCKER_ISOLATION.md custom-image and runtime-inject sections) that install their own unpinned uv and build a venv in the bind-mounted workspace. (2) The host driver: local path in src/coder_eval/sandbox.py:808-840, which uses whatever uv is on the host, with the cache on a different filesystem from the sandbox. DOCKER_ISOLATION.md:18 describes uv as part of the 'pinned toolchain' but gives no guidance for images that bring their own uv. (trigger: docker/Dockerfile)

Tests:

• 🔵 test_uv_pinned_in_agent_image only checks the Dockerfile text. No check confirms that the built image actually contains uv == UV_VERSION (for example, a uv --version assert in the docker-publish/release build or the image smoke test). Such a check would catch a failed versioned download or an installer that ignores the version. (trigger: tests/test_image_from_dockerfiles.py) (restates: Axis 6: A failed versioned uv download makes the install layer exit 0 and the build later fails with "uv: not found")
• 🔵 The comment's bump rule ('Bump deliberately, after timing an install into a bind-mounted dir') is a process note only. No benchmark, make target or opt-in test times a warm-cache uv pip install into a bind mount against a threshold. The next person who bumps UV_VERSION has no tool to do the timing, so the same regression can come back on a deliberate bump. (trigger: docker/Dockerfile)

Harness & Lint Improvements

Static checks (lint / type):

• [ce-lint] CE067 (Dockerfile pipefail): any RUN in docker/Dockerfile* that pipes into a shell or another command (a single |, not ||) must sit after a SHELL ["/bin/bash", "-o", "pipefail", "-c"] in the same build stage, or must start with set -o pipefail. This is the same check as hadolint DL4006. Dockerfiles are not Python ASTs, so add it as a stdlib text-scan @pytest.mark.lint class in tests/test_custom_lint.py (the whole-tree / doc-surface slot), not in tests/lint/runner.py. Note: a pipefail-only fix still lets curl -f fail with no output, sh read an empty script and exit 0. So the rule should also accept the download-to-file form (curl -o f && sh f) as compliant. Prevents: Finding 3 (docker/Dockerfile:56, a 404 or 5xx on the versioned uv download exits 0 and the build later fails with uv: not found). It also flags the same unchecked pipes that exist today at docker/Dockerfile:34 (nodesource curl | bash) and docker/Dockerfile.runtime:38 (uv curl | sh).
• [ce-lint] CE068 (Dockerfile tool pins, table-driven): one rule over every ARG <NAME>_VERSION=<v> in docker/Dockerfile and docker/Dockerfile.runtime. It asserts that (a) the value is not latest or empty, (b) at least one RUN line references ${<NAME>_VERSION}, so the install cannot drift from the pin, (c) an ARG name declared in both Dockerfiles has the same value in both, and (d) a known installer URL or package (astral.sh/uv/install.sh, npm install -g <pkg>) is never fetched without a version segment. It uses one module-level _arg_pins(df_text) -> dict[str, str] parser. Delete before you guard: this rule replaces the three hand-written per-tool tests in tests/test_image_from_dockerfiles.py (test_claude_code_version_pin_matches_framework, test_pi_cli_baked_and_pinned, test_uv_pinned_in_agent_image), so the duplicated ARG-parsing loop goes away and does not get a shared helper. Add it as a @pytest.mark.lint class in tests/test_custom_lint.py, or keep it in tests/test_image_from_dockerfiles.py as one parametrized test. A fourth pinned tool then needs no new code. Prevents: Finding 1 (the third copy of the inline ARG <NAME>= pin loop at tests/test_image_from_dockerfiles.py:495-502, 523-530 and 473-478, plus the second copy of the framework-Dockerfile path expression). Check (d), and check (c) for any shared ARG, also flag an unpinned uv install that remains in the parallel docker/Dockerfile.runtime:38, which the PR may not have pinned. The same would apply to any later tool added to only one image.
• [ce-lint] Extend make docs-budget (tests/lint/prose_budget.py _ROOTS, which today is only src/coder_eval and tests) to scan the # comment runs in docker/Dockerfile* and docker/*.sh with the existing _COMMENT_RUN_LINES = 8 run cap and _RUN_BLANK_BRIDGE. Dockerfile comments are # own-line comments, so the existing run logic applies unchanged and only needs a non-tokenize line reader for non-Python files. The 'history versus contract' judgement (benchmark numbers, incident dates, an ambiguous 0.12.1 image) is semantic and stays a review item. The run cap enforces the shape, which pushes the paragraph into .claude/notes/ behind a Rationale: pointer. Prevents: Finding 2 (the 10-line uv pin comment at docker/Dockerfile:45-54 is over the 8-line cap but was not checked, because docker/ is outside the budget's roots). The test-docstring half of finding 2 (tests/test_image_from_dockerfiles.py:489-494) is under the 150-word docstring cap, so only review can catch it. If CE068 replaces that test, the docstring is deleted with it.

Harness improvements (not statically reachable):

• Add a post-build image-contract check (a make docker-image-verify target, run in docker-publish.yml and in pr-checks.yml when docker/** changes). For each ARG <NAME>_VERSION that CE068 parses, it runs the tool's --version inside the built coder-eval-agent and coder-eval-runtime images and asserts that the output contains the pinned value. This is the generic form of the && uv --version | grep -F "${UV_VERSION}" that finding 3 recommends. Why not static: Only a built image shows which binary the installer actually put on PATH. A static scan can confirm that the URL contains the version, but it cannot see an installer that ignores it, redirects to latest, or fails silently. Prevents: Finding 3 (the build reports success, or fails at the wrong step, when the pinned uv version cannot be fetched), and any drift between a pinned ARG and the installed binary.
• Record the uv bind-mount install benchmark (0.3s on 0.12.10 versus 10-14s on 0.12.11+, and 60-130s on CI) and the incident in a .claude/notes/ entry. Add a make docker-uv-bench target that times a uv pip install of the project lockfile into a bind-mounted directory in the agent image. The Dockerfile comment then states only the contract ('a uv bump requires running make docker-uv-bench') plus a Rationale: pointer. Why not static: The regression is a cross-filesystem copy-performance cliff. It only appears at runtime on a real bind mount on the CI host, so no source pattern shows it. Prevents: Finding 2 (history and benchmark data inline in docker/Dockerfile:45-54), and an unbenchmarked future uv bump bringing back the 40x slowdown that the pin exists to prevent.

Top 5 Priority Actions

1. No finding can change a task's score or final_status for identical agent output, so start with the build risk: at docker/Dockerfile:56, add SHELL ["/bin/bash", "-o", "pipefail", "-c"] before the line (or download install.sh to a file first) and append && uv --version | grep -F "${UV_VERSION}", so a 404 or network failure on the pinned uv makes this step fail instead of the later 'uv: not found' at docker/Dockerfile:107.
2. At tests/test_image_from_dockerfiles.py:495-502 (also 523-530 and the nested _pin at 473-478), move the pin parsing to a module-level _arg_pin(df_text, name) -> str | None and add _framework_dockerfile_text() next to _runtime_dockerfile() at line 416, so that the three copies become one and the next pinned tool does not add a fourth.
3. At docker/Dockerfile:45-54, cut the 10-line comment down to the contract (uv is pinned because the copy into the bind-mounted workspace is about 40x slower from 0.12.11, and a bump must time an install into a bind-mounted dir), and move the benchmark timings and the 2026-09-12 incident to a .claude/notes/ entry or the commit message.
4. At tests/test_image_from_dockerfiles.py:489-494, remove the history from the test docstring ('moved the image from uv 0.12.9 to 0.12.12 on a rebuild') and keep only the contract in its last sentence; where a release must be named, write 'coder_eval v0.12.1' so that no one reads it as a uv version.
5. Add a test or a .claude/harness-candidates.md entry that fails when a Dockerfile RUN curl ... | sh runs without pipefail, so this class of silent install failure (the root cause at docker/Dockerfile:56, which was there before the PR) cannot come back on another pinned tool.

---

Stats: 0 🔴 · 0 🟠 · 1 🟡 · 2 🔵 across 8 axes reviewed.

[uipreliga]

fix what you agree with and 🚢