ci(gitlab): streamline eic manifest DAG dependencies - #381
Open
wdconinc wants to merge 1 commit into
Open
Conversation
Contributor
There was a problem hiding this comment.
🟡 Not ready to approve
The new matrix needs selector appears to be mis-specified (array-vs-scalar mismatch) and eic-manifest doesn’t currently mirror .nightly rules, both of which can break artifact matching / DAG satisfaction and trigger the new “No metadata files found” failure.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.
Pull request overview
This PR refactors the GitLab CI DAG around eic-manifest so manifest jobs can start as soon as their corresponding eic build shard finishes, reducing unnecessary waiting and duplication in the pipeline configuration.
Changes:
- Converted
eic-manifestinto aparallel:matrixjob aligned with the existingeicmatrix via a shared YAML anchor. - Added matrix-aware
needswiring so eacheic-manifestshard depends on its matchingeicshard and processes only its ownbuild-metadata-${BUILD_IMAGE}${ENV}-*.jsonfiles. - Updated downstream jobs to depend only on the specific manifest shard they require via anchored
needsentries.
File summaries
| File | Description |
|---|---|
.gitlab-ci.yml |
Reworks eic-manifest into a matrix job and rewires needs dependencies to be shard-specific using anchors/aliases. |
Review details
Suppressed comments (1)
.gitlab-ci.yml:488
- The
needs:parallel:matrixselector for the upstreameicjob uses single-element arrays (e.g.BUILD_IMAGE: ['…']). The upstreameicmatrix defines these variables as scalars, so this type mismatch can prevent GitLab from matching the correcteicshard and downloading its artifacts, which will make this shard hit the new "No metadata files found" error.
- BUILD_IMAGE: ['$[[ matrix.BUILD_IMAGE ]]']
ENV: ['$[[ matrix.ENV ]]']
BUILD_TYPE: ['$[[ matrix.BUILD_TYPE ]]']
BUILDER_IMAGE: ['$[[ matrix.BUILDER_IMAGE ]]']
RUNTIME_IMAGE: ['$[[ matrix.RUNTIME_IMAGE ]]']
- Files reviewed: 1/1 changed files
- Comments generated: 1
- Review effort level: Lite
We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.
Comment on lines
474
to
478
| eic-manifest: | ||
| parallel: | ||
| matrix: *eic_matrix | ||
| extends: .docker | ||
| stage: eic-manifest |
Refactor GitLab CI DAG wiring so eic-manifest runs as a matrix aligned with eic and starts per-container as soon as its matching build shard completes. Restrict each manifest shard to its own metadata files and keep benchmark and deploy jobs dependent only on the specific manifest shard they require. Use YAML anchors to remove duplicated matrix and needs blocks so eic and eic-manifest stay synchronized and eic_ci/eic_xl manifest selectors are reused consistently. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
wdconinc
force-pushed
the
wdconinc-optimize-gitlab-dag
branch
from
August 19, 2026 16:49
522aa30 to
4f9e9c2
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.
Suppressed comments (2)
.gitlab-ci.yml:524
- The new error is helpful, but it would be easier to debug if it included the exact glob/pattern being searched so the log shows what the job expected to find.
if [ "${matched}" -eq 0 ]; then
echo "ERROR: No metadata files found for ${BUILD_IMAGE}${ENV}."
exit 1
.gitlab-ci.yml:483
needsmarks theeicdependency asoptional: true, but this job now hard-fails when no matching metadata files exist. Ifeicis ever skipped by rules,eic-manifestwould still start and then fail due to missing artifacts. Making theeicneed non-optional keeps the DAG strict and avoids a confusing failure mode.
needs:
- version
- job: eic
optional: true
parallel:
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This streamlines the GitLab pipeline DAG so jobs start as soon as their required artifacts are ready, instead of waiting on unrelated shards.
What changed
eic-manifestinto a matrix job aligned with theeicmatrix.needsfrom eacheic-manifestshard to its matchingeicshard.build-metadata-${BUILD_IMAGE}${ENV}-*.jsonso each shard handles only its own image metadata.needsso benchmark and deploy jobs depend only on the specific manifest shard they require.Duplication reduction and synchronization
eicmatrix and reused it ineic-manifest.needsselectors foreic_ciandeic_xlmanifest dependencies.Notes