[deckhouse-cli] Fix --include-module ignoring semver constraints#341
Open
Glitchy-Sheep wants to merge 14 commits intomainfrom
Open
[deckhouse-cli] Fix --include-module ignoring semver constraints#341Glitchy-Sheep wants to merge 14 commits intomainfrom
Glitchy-Sheep wants to merge 14 commits intomainfrom
Conversation
Signed-off-by: Roman Berezkin <roman.berezkin@flant.com>
It's obvious from the flags logic itself, and it simplifies the following code. Signed-off-by: Roman Berezkin <roman.berezkin@flant.com>
Signed-off-by: Roman Berezkin <roman.berezkin@flant.com>
… zero value (nil) + test for the behavior + names refactor Signed-off-by: Roman Berezkin <roman.berezkin@flant.com>
Signed-off-by: Roman Berezkin <roman.berezkin@flant.com>
- `ListTags` for a module's tag list now runs only when the constraint is semver. Exact-tag and blacklist-mode modules take filter branches that never read `Module.Releases`, so the call was wasted work. - For a default `d8 mirror pull` (no `--include-module`), per-module `ListTags` drops to zero across the registry. - Adds tests verifying the skip for exact-tag and blacklist cases, plus a counter-check that semver constraint still triggers the call. Signed-off-by: Roman Berezkin <roman.berezkin@flant.com>
Signed-off-by: Roman Berezkin <roman.berezkin@flant.com>
Signed-off-by: Roman Berezkin <roman.berezkin@flant.com>
…ration) Signed-off-by: Roman Berezkin <roman.berezkin@flant.com>
- `Module(name).ListTags` now warns and falls back to channel-only versions on `client.ErrImageNotFound`, matching the policy of `validateModulesAccess`. Other errors keep the existing fail-fast behavior. Signed-off-by: Roman Berezkin <roman.berezkin@flant.com>
Signed-off-by: Roman Berezkin <roman.berezkin@flant.com>
- 8 new phase methods make the orchestrator read top-to-bottom as a plan. - Each phase preserves its error policy: fail-fast for required pulls, debug-log-and-continue for release-version and internal-digest pulls where missing artifacts are expected. - All 10 phases live right after the orchestrator (`pullVexImages` and `findVexImage` moved into the cluster), so the call order is visible in file order. Signed-off-by: Roman Berezkin <roman.berezkin@flant.com>
Signed-off-by: Roman Berezkin <roman.berezkin@flant.com>
- `TestPullModules_*` names match the public entry point each test exercises. - The per-module `ListTags` policy is covered by two table-driven tests: call count and error handling. - Assertions read the download list through the public `Module(name)` accessor. - File is split into named sections - fixtures, builders, assertions, test doubles - with consistent helper names. Signed-off-by: Roman Berezkin <roman.berezkin@flant.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
[mirror] Fix --include-module ignoring semver constraints during pull
Summary
d8 mirror pull --include-module <name>@<semver>was pulling only the channel version, not the full semver range.The fix gives the version filter the actual tag list from the registry, so the constraint matches every fitting version.
Problem
pullSingleModulebuilt&Module{Releases: nil}before callingFilter.VersionsToMirror. The semver branch of the filter iteratesReleases, so it had nothing to match.v1.45.2(the channel version), missingv1.40.0throughv1.44.0that all matched^1.40.0in the registry.@=v1.40.0) were not affected - they don't readReleases.Releasesby hand, so they didn't catch the integration gap.Fix
pullSingleModulenow callsmodulesService.Module(name).ListTags(ctx)and stores the result onModule.Releases. The semver branch sees every published version.hasConstraint && !constraint.IsExact(), so it is skipped for exact-tag whitelist and for blacklist-mode modules that don't have a constraint. A default pull (no--include-module) makes zero extra requests across the registry.Behavior Map:
Registry tags:
v1.39.0, v1.40.0, v1.40.1, v1.41.0, v1.42.0, v1.43.0, v1.45.2. Channels →v1.45.2.Legend: ✅ correct, ❌ broken
console(no@)>=0.0.0)v1.45.2❌@1.40.0(implicit^)>=1.40.0 <2.0.0v1.45.2❌@^1.40.0>=1.40.0 <2.0.0v1.45.2❌@~1.40.0>=1.40.0 <1.41.0v1.45.2❌v1.40.0, v1.40.1+ channel ✅@>=1.40.0 <1.43.0v1.45.2❌@>1.42.0>1.42.0v1.45.2❌v1.43.0, v1.45.2✅@<1.41.0<1.41.0v1.45.2❌v1.39.0, v1.40.0, v1.40.1+ channel ✅@=v1.40.0v1.40.0+ 5 channel aliases ✅v1.40.0+ 5 channel aliases ✅ unchanged@=v1.40.0+stablestableonlyv1.40.0+ stable alias ✅v1.40.0+ stable alias ✅ unchangedNotes:
--include-module foomaps to>=0.0.0internally → all historical versions of the module. Before the fix this was masked (only channel snapshot pulled). After the fix, real registries with ~100+ historical tags produce dramatically larger bundles.--include-moduleand--exclude-moduleare now mutually exclusive at the cobra flag level. Before, both could be passed and whitelist silently won. The flags logic was simplified.c/ok/e/scinFilter.VersionsToMirrorandFilter.ShouldMirrorReleaseChannelstoconstraint/hasConstraint/exact/isExactTag/semver/isSemverfor clarity.internal/mirror/modules/modules.go- divided monolithic hard to read function into modular functions.Before / After
Showcase:
--include-module console@1.40.0(semver^1.40.0) - the canonical case from the bug report. Other syntax variants (console,console@~1.40.0,console@=v1.40.0) were also verified manually with no regression.Before: only the channel snapshot (

v1.45.2). The^1.40.0constraint had no effect.After: every tag in the registry that matches

^1.40.0- 33 versions fromv1.40.0throughv1.45.2.Test plan
Covered in
internal/mirror/modules/pull_versions_test.go: semver matching (caret / tilde / range), multi-module independence, the per-moduleListTagsskip optimization, and both error-paths (ErrImageNotFoundskipped vs other errors propagated).