Skip to content

fix(harbor): accept artifact destination and exclude from Harbor task.toml - #579

Open
ayushnangia wants to merge 3 commits into
hud-evals:mainfrom
ayushnangia:fix/harbor-artifact-interop
Open

fix(harbor): accept artifact destination and exclude from Harbor task.toml#579
ayushnangia wants to merge 3 commits into
hud-evals:mainfrom
ayushnangia:fix/harbor-artifact-interop

Conversation

@ayushnangia

@ayushnangia ayushnangia commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Problem

harbor.adapt() rejects Harbor tasks that Harbor itself accepts. Harbor's ArtifactConfig (harbor 0.20.0, harbor/models/task/config.py) has two fields the adapter's Artifact model forbids:

[[artifacts]]
source = "/app/outputs"
destination = "results/outputs"   # extra_forbidden -> "not a valid Harbor task"
exclude = ["*.tmp", "cache"]      # extra_forbidden -> "not a valid Harbor task"

Any task directory using either field fails adaptation outright.

Fix

destination — accepted and preserved through the task args. No runtime behavior change: Harbor documents that verifier-side placement never depends on destination (it only controls host placement under the trial's artifacts dir), and this runtime already re-mounts artifacts at their source path, which matches Harbor's verifier-side contract. Validation mirrors Harbor's: relative, forward slashes, no .., manifest.json reserved — so tasks Harbor would reject still fail adaptation loudly.

exclude — implemented, not just accepted. Harbor applies these as tar --exclude flags when downloading directory artifacts, so silently ignoring them would hand the verifier files Harbor's verifier never sees. env.py now prunes matching entries after staging a directory artifact, following GNU tar exclusion semantics (unanchored: a pattern matches any run of trailing path components; a matched directory prunes its subtree).

Artifact source intentionally still requires an absolute path: this runtime mounts artifacts back at their source path for the verifier, which needs an absolute anchor. Harbor itself requires absolute sources for sidecar services; relative sources for the main container are a separate discussion.

Testing

  • uv run pytest hud/integrations/harbor/tests/test_contract.py hud/integrations/harbor/tests/test_harbor.py — 67 passed (new: destination/exclude round-trip into task args; Harbor-invalid destinations fail adaptation)
  • uv run pytest -m integration -k "artifact or restores_image_paths or excluded_entries" — 5 passed, including a new end-to-end test: a directory artifact with exclude = ["*.tmp", "cache"] is staged for a separate verifier without the excluded entries (nested matches included), reward 1.0
  • ruff format --check, ruff check, ty check — clean

The exclusion matcher follows GNU tar's documented --exclude behavior (unanchored glob over component boundaries + subtree pruning); byte-for-byte parity with every tar edge case is not claimed.


Note

Medium Risk
Changes artifact staging and symlink handling in the verifier collect path; incorrect exclude semantics could hide or leak files to graders, but scope is limited to Harbor integration.

Overview
Harbor tasks that declare destination or exclude on artifacts no longer fail harbor.adapt(); those fields are modeled on Artifact with Harbor-aligned validation (relative destination, no .., reserved manifest.json).

At collect time, directory artifacts are staged with copytree(..., symlinks=True), then exclude patterns prune matching paths via new prune_excluded (GNU tar–style unanchored globs and subtree removal). Symlink checks still reject symlink roots and any symlinks left in the staged tree after pruning.

Contract and Docker integration tests cover round-trip args, invalid destinations, sidecar symlink roots, and end-to-end grading when exclude drops *.tmp and cache from verifier-visible artifacts.

Reviewed by Cursor Bugbot for commit a537e06. Bugbot is set up for automated code reviews on this repo. Configure here.

Harbor ArtifactConfig allows destination (host placement, no verifier-side
effect) and exclude (tar --exclude patterns applied when downloading
directory artifacts). The adapter rejected both with extra_forbidden, so
valid Harbor tasks failed to adapt. Accept destination with Harbor's own
validation, and prune excluded entries when staging directory artifacts so
the verifier sees what Harbor's verifier would see.
Comment thread hud/integrations/harbor/env.py

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 2f9b01b. Configure here.

Comment thread hud/integrations/harbor/env.py
@ayushnangia

Copy link
Copy Markdown
Contributor Author

On the second finding (prune follows symlink artifact roots) — fixed in a537e06 on the branch.

The reorder that caused this is now inverted where it matters: collect() checks target.is_symlink() and raises before artifact["exclude"] and target.is_dir() runs prune_excluded, so on the docker cp path a directory-symlink artifact root is rejected up front and pruning never traverses through the link target. Same order for the copy_artifact path (which additionally rejects root symlinks internally), and the post-prune rglob sweep still catches symlinks anywhere inside real directories.

The first finding's fix (staging made inert + filter-before-symlink-rejection, 2f9b01b) is unchanged. Thanks for the catch on the interaction — the follow-through order matters more than the excludes themselves.

jdchawla29 added a commit that referenced this pull request Aug 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant