Skip to content

BUILD-11964: Harden config-uv dependency cache keys#320

Draft
bwalsh434 wants to merge 4 commits into
masterfrom
feat/bwalsh/BUILD-11937-partition-uv-cache-key-by-workflow
Draft

BUILD-11964: Harden config-uv dependency cache keys#320
bwalsh434 wants to merge 4 commits into
masterfrom
feat/bwalsh/BUILD-11937-partition-uv-cache-key-by-workflow

Conversation

@bwalsh434

@bwalsh434 bwalsh434 commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Summary

  • derive uv cache keys from dependency content only, allowing reuse across workflows
  • support an explicit dependency glob for jobs that install multiple uv projects
  • warn and disable caching when no dependency files match instead of freezing uv-Linux-

Predicted savings

Approximately 0 TiB/month directly. This is the shared prerequisite and safety guard for BUILD-11965 through BUILD-11969. The measurable savings occur when those consumers supply correct project or lockfile scope.

Removing workflow-name partitioning avoids duplicate caches and also avoids the Unicode workflow-name failure independently confirmed in Semsitter.

Rollout and confirmation

  1. Merge and release this change on the v1 tag.
  2. Run each consumer PR twice.
  3. Confirm the first run saves a non-empty dependency-hash key and the second restores the exact key.
  4. After consumer rollout, compare 7-14 day pre/post repository uv bytes and repeated immutable wheel paths in the uv caching notebook.

Success for this PR is no new empty keys and correct key invalidation when scoped lockfiles change. Savings are attributed and verified in each consumer PR.

Test plan

  • Parse config-uv/action.yml
  • Run git diff --check
  • Run focused ShellSpec suite: 5 examples, 0 failures
  • Canary a root uv project
  • Canary a nested uv project with working-directory
  • Canary a multi-project job with cache-dependency-glob

The full repository ShellSpec suite currently has two unrelated failures: one existing SCA mock expectation and one missing local check-jsonschema executable.

config-uv's GitHub Actions cache key was shared across all workflows
in a repo (uv-${runner.os}-${hash}), unlike config-poetry which scopes
its key per workflow. Repos with multiple concurrent workflows thrash
a single shared uv cache bucket against GitHub's 10GB per-repo cache
limit, causing repeated cache misses and re-downloads from Artifactory.

Datadog logs show uv re-fetching identical content-hashed wheel URLs
up to 184 times over 14 days (avg 69/wheel) vs poetry's max of 4
(avg 2.2/wheel), and uv's Artifactory bandwidth is up 345.7% over 90
days -- the classic signature of a cache that isn't holding.

This mirrors config-poetry's existing sanitize-workflow-name + scoped
cache key pattern exactly.
@hashicorp-vault-sonar-prod

hashicorp-vault-sonar-prod Bot commented Jul 13, 2026

Copy link
Copy Markdown

BUILD-11937

@bwalsh434

Copy link
Copy Markdown
Contributor Author

Gitar fix CI

@gitar-bot

gitar-bot Bot commented Jul 13, 2026

Copy link
Copy Markdown

Gitar fix CI

On it — working on fixing the CI failure now.

Co-authored-by: Brian Walsh <20115224+bwalsh434@users.noreply.github.com>
@bwalsh434

Copy link
Copy Markdown
Contributor Author

Will be telling Claude about the gh-action_cache, as I presume these cases are using that with S3 and not thrashing against the 10GB Limit of the GHA Cache.

@bwalsh434 bwalsh434 changed the title BUILD-11937: partition config-uv cache key by workflow name BUILD-11937: Partition config-uv cache key by workflow name Jul 14, 2026
Comment thread config-uv/action.yml
Comment on lines +106 to +111
dependency_hash="$DIRECTORY_DEPENDENCY_HASH"
dependency_source="working-directory '${WORKING_DIRECTORY}'"
if [[ -n "$CACHE_DEPENDENCY_GLOB" ]]; then
dependency_hash="$GLOB_DEPENDENCY_HASH"
dependency_source="cache-dependency-glob '${CACHE_DEPENDENCY_GLOB}'"
fi

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Edge Case: cache-dependency-glob silently overrides working-directory hash

When both working-directory and a non-empty cache-dependency-glob are provided, the glob hash unconditionally replaces the working-directory hash, so the working-directory's uv.lock/pyproject.toml are excluded from the cache key. This is fine for the documented multi-project use case (working-directory left at '.'), but a user who sets both intentionally would get a key that ignores their working-directory lockfiles without any warning. Consider documenting the precedence explicitly, or combining both file sets into the hash when both inputs are set.

Was this helpful? React with 👍 / 👎

Comment thread config-uv/action.yml
Comment on lines +94 to +108
- name: Resolve uv cache key
id: uv-cache-key
if: steps.config-uv-completed.outputs.skip != 'true' && inputs.disable-caching != 'true'
shell: bash
env:
CACHE_DEPENDENCY_GLOB: ${{ inputs.cache-dependency-glob }}
DIRECTORY_DEPENDENCY_HASH:
${{ hashFiles(format('{0}/uv.lock', inputs.working-directory),
format('{0}/pyproject.toml', inputs.working-directory)) }}
GLOB_DEPENDENCY_HASH: ${{ hashFiles(inputs.cache-dependency-glob) }}
WORKING_DIRECTORY: ${{ inputs.working-directory }}
run: |
dependency_hash="$DIRECTORY_DEPENDENCY_HASH"
dependency_source="working-directory '${WORKING_DIRECTORY}'"
if [[ -n "$CACHE_DEPENDENCY_GLOB" ]]; then

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Quality: New cache-key resolution bash logic has no test coverage

The 'Resolve uv cache key' step adds non-trivial branching (glob-vs-working-directory hash selection and the empty-hash 'disable caching + warn' path), but spec/config-uv_spec.sh only exercises uv_config.sh, not the composite action's embedded bash. The PR test plan explicitly notes the shell specs are unaffected, so these branches ship untested. Consider extracting the resolution logic into a sourced script that shellspec can cover, or adding a targeted test for the empty-hash and glob paths.

Was this helpful? React with 👍 / 👎

@bwalsh434 bwalsh434 changed the title BUILD-11937: Partition config-uv cache key by workflow name BUILD-11964: Harden config-uv dependency cache keys Jul 15, 2026
@gitar-bot

gitar-bot Bot commented Jul 16, 2026

Copy link
Copy Markdown
Code Review 👍 Approved with suggestions 0 resolved / 2 findings

Partitions config-uv cache keys by workflow name to resolve cache thrashing, though the new cache-dependency-glob logic overrides working-directory hashes and currently lacks spec coverage.

💡 Edge Case: cache-dependency-glob silently overrides working-directory hash

📄 config-uv/action.yml:106-111

When both working-directory and a non-empty cache-dependency-glob are provided, the glob hash unconditionally replaces the working-directory hash, so the working-directory's uv.lock/pyproject.toml are excluded from the cache key. This is fine for the documented multi-project use case (working-directory left at '.'), but a user who sets both intentionally would get a key that ignores their working-directory lockfiles without any warning. Consider documenting the precedence explicitly, or combining both file sets into the hash when both inputs are set.

💡 Quality: New cache-key resolution bash logic has no test coverage

📄 config-uv/action.yml:94-108

The 'Resolve uv cache key' step adds non-trivial branching (glob-vs-working-directory hash selection and the empty-hash 'disable caching + warn' path), but spec/config-uv_spec.sh only exercises uv_config.sh, not the composite action's embedded bash. The PR test plan explicitly notes the shell specs are unaffected, so these branches ship untested. Consider extracting the resolution logic into a sourced script that shellspec can cover, or adding a targeted test for the empty-hash and glob paths.

🤖 Prompt for agents
Code Review: Partitions `config-uv` cache keys by workflow name to resolve cache thrashing, though the new `cache-dependency-glob` logic overrides working-directory hashes and currently lacks spec coverage.

1. 💡 Edge Case: cache-dependency-glob silently overrides working-directory hash
   Files: config-uv/action.yml:106-111

   When both `working-directory` and a non-empty `cache-dependency-glob` are provided, the glob hash unconditionally replaces the working-directory hash, so the working-directory's uv.lock/pyproject.toml are excluded from the cache key. This is fine for the documented multi-project use case (working-directory left at '.'), but a user who sets both intentionally would get a key that ignores their working-directory lockfiles without any warning. Consider documenting the precedence explicitly, or combining both file sets into the hash when both inputs are set.

2. 💡 Quality: New cache-key resolution bash logic has no test coverage
   Files: config-uv/action.yml:94-108

   The 'Resolve uv cache key' step adds non-trivial branching (glob-vs-working-directory hash selection and the empty-hash 'disable caching + warn' path), but spec/config-uv_spec.sh only exercises uv_config.sh, not the composite action's embedded bash. The PR test plan explicitly notes the shell specs are unaffected, so these branches ship untested. Consider extracting the resolution logic into a sourced script that shellspec can cover, or adding a targeted test for the empty-hash and glob paths.

Options

Auto-apply is off → Gitar will not commit updates to this branch.
Display: compact → Showing less information.

Comment with these commands to change the behavior for this request:

Auto-apply Compact
gitar auto-apply:on         
gitar display:verbose         

Was this helpful? React with 👍 / 👎 | Gitar

@sonarqubecloud

Copy link
Copy Markdown

Comment thread config-uv/action.yml
Comment on lines +21 to +23
cache-dependency-glob:
description: Glob of dependency files for jobs that install more than one uv project
default: ''

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add to the README. Point in the PR description to the usages.
Isn't the default "inputs.working-directory/uv.lock"?
Why do we need this instead of using the working-directory?

Comment thread config-uv/action.yml
CACHE_DEPENDENCY_GLOB: ${{ inputs.cache-dependency-glob }}
DIRECTORY_DEPENDENCY_HASH:
${{ hashFiles(format('{0}/uv.lock', inputs.working-directory),
format('{0}/pyproject.toml', inputs.working-directory)) }}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why pyproject.toml? Maybe a question for @hedinasr since it was in "Cache uv dependencies".

Comment thread config-uv/action.yml
DIRECTORY_DEPENDENCY_HASH:
${{ hashFiles(format('{0}/uv.lock', inputs.working-directory),
format('{0}/pyproject.toml', inputs.working-directory)) }}
GLOB_DEPENDENCY_HASH: ${{ hashFiles(inputs.cache-dependency-glob) }}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rename GLOB_DEPENDENCY_HASH -> CACHE_DEPENDENCY_HASH

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants