v3 - #225
Draft
alexey-igrychev wants to merge 267 commits into
Draft
Conversation
Signed-off-by: Dmitry Mordvinov <dmitry.mordvinov@flant.com>
Signed-off-by: Dmitry Mordvinov <dmitry.mordvinov@flant.com>
Signed-off-by: Dmitry Mordvinov <dmitry.mordvinov@flant.com> # Conflicts: # pkg/deploy/helm_for_werf_helm/chart_extender_for_werf_helm/helpers_for_werf_helm/service_values.go
Signed-off-by: Dmitry Mordvinov <dmitry.mordvinov@flant.com>
Signed-off-by: Dmitry Mordvinov <dmitry.mordvinov@flant.com>
Signed-off-by: Dmitry Mordvinov <dmitry.mordvinov@flant.com>
Signed-off-by: Dmitry Mordvinov <dmitry.mordvinov@flant.com> # Conflicts: # go.mod # go.sum
Remove all Ansible-related code, config parsing, CLI flags, tests, Stapel container packages, and documentation (EN/RU) for the Ansible builder in Stapel build instructions. Only the Shell builder remains. Deleted: - pkg/build/builder/ansible.go, ansible_assets.go, ansible_assets_data.go - pkg/build/builder/ansible/ (Python scripts, static embed) - pkg/config/ansible.go, ansible_task.go, raw_ansible.go, raw_ansible_task.go - test/legacy_e2e/suites/ansible/ (16 test files) - stapel/omnibus/config/software/ansible.rb Modified: - Removed Ansible branches from builder selection, config parsing, conveyor checks, and CLI env annotations - Relocated shared Extra struct and inlined mkdirP in shell.go - Cleaned Stapel Dockerfile and omnibus dependency - Updated all EN/RU docs and data YAML files Signed-off-by: Aleksei Igrychev <aleksei.igrychev@palark.com>
Remove deprecated artifact image type from werf codebase: - Rename ArtifactExport → Export, rawArtifactExport → rawExport - Delete StapelImageArtifact struct and artifact parsing logic - Remove artifact/fromArtifact/ArtifactName fields from config structs - Remove artifact directive references from parser and validator - Clean up build pipeline (dependencies, imports, image checks) - Update test fixtures to use image with final: false - Remove artifact references from documentation and data files BREAKING CHANGE: The artifact directive is no longer supported. Use image with final: false instead. Signed-off-by: Aleksei Igrychev <aleksei.igrychev@palark.com>
Signed-off-by: Aleksei Igrychev <aleksei.igrychev@palark.com>
Signed-off-by: Aleksei Igrychev <aleksei.igrychev@palark.com>
Signed-off-by: Aleksei Igrychev <aleksei.igrychev@palark.com>
Signed-off-by: Aleksei Igrychev <aleksei.igrychev@palark.com>
Signed-off-by: Aleksei Igrychev <aleksei.igrychev@palark.com>
…gest Signed-off-by: Aleksei Igrychev <aleksei.igrychev@palark.com>
Signed-off-by: Aleksei Igrychev <aleksei.igrychev@palark.com>
…upport removal Signed-off-by: Aleksei Igrychev <aleksei.igrychev@palark.com>
… API Replace forked werf/3p-docker-buildx with upstream docker/buildx, migrating from cobra-based `buildx build --load` CLI emulation to direct commands.RunBuild() programmatic API call. - Remove legacy DOCKER_BUILDKIT=0 code path and useBuildx flag - Extract image ID from SolveResponse instead of temp metadata file - Detach docker CLI subcommands from root to fix flag leaking - Add unit test validating required docker CLI commands at build time - Regenerate CLI reference docs for kubectl v0.35 Signed-off-by: Aleksei Igrychev <aleksei.igrychev@palark.com>
…int to v2 Upgrade docker/buildx from v0.28.0 to v0.33.0, requiring Go 1.25. Transitive dependency upgrades: - docker/docker v25 -> v28: migrate deprecated type aliases - moby/buildkit v0.17 -> v0.29: update instructions.Parse, shell.Lex APIs - distribution/reference v0.5 -> v0.6: switch from docker/distribution - containers/image,storage,common -> go.podman.io module paths - containers/buildah: drop werf/3p-buildah fork, use upstream v1.43.0 - fsouza/go-dockerclient v1.12.2 -> v1.12.4 - go.podman.io/storage: upgrade for filepath-securejoin v0.6.x compat - werf/3p-oras, oras-go: update for new module paths - golangci-lint v1 -> v2 with config migration Signed-off-by: Aleksei Igrychev <aleksei.igrychev@palark.com>
- Drop parentID resolution, use WerfBaseImageIDLabel only - Drop Names from buildah ImagesOptions, use only Filters - Remove unused ClaimTargetPlatforms methods and related checks - Remove RootlessStoragePath and HasCapSysAdmin calls dropped in new storage version Signed-off-by: Aleksei Igrychev <aleksei.igrychev@palark.com>
containerd/containerd v1.7.28 defers root directory creation in local.NewStore (lazy sync.OnceValue) unlike v1.7.14 which eagerly called os.MkdirAll. This caused NewOCIStore to fail writing the oci-layout file when the helm cache directory didn't exist yet. Signed-off-by: Aleksei Igrychev <aleksei.igrychev@palark.com>
BuildKit is now the only Dockerfile builder for the Docker backend, so the separate vanilla-docker and buildkit-docker test modes are redundant. Unify them into a single "docker" mode, merge duplicate test entries, and remove the vanilla-docker conditional that restricted secrets tests to stapel-only builds. Signed-off-by: Aleksei Igrychev <aleksei.igrychev@palark.com>
Provenance attestations cause buildx to produce index manifests instead of regular image manifests. Current codebase cannot read labels from index manifests, which causes missing werf-stage-content-digest label on cached stages. Signed-off-by: Aleksei Igrychev <aleksei.igrychev@palark.com>
werf dismiss hangs on "Waiting for resources elimination" in nelm's LegacyReleaseUninstall, causing e2e tests to stall. Signed-off-by: Aleksei Igrychev <aleksei.igrychev@palark.com>
Signed-off-by: Evgeniy Frolov <evgeniy.frolov@flant.com>
Signed-off-by: Aleksei Igrychev <aleksei.igrychev@palark.com>
…alidation Scratch was incorrectly marked as FromExternal and routed through ImageFromRegistryAsBaseImage, relying on scattered string comparisons to special-case it. Introduce a dedicated ScratchBaseImage type with proper cases in SetupBaseImage, FetchBaseImage, and mapStapelConfig. Also add config-level guards: reject fromLatest with scratch, reject import from scratch, reserve "scratch" as image name, and skip external-image tag requirement for scratch. Signed-off-by: Aleksei Igrychev <aleksei.igrychev@palark.com>
Signed-off-by: Dmitry Mordvinov <dmitry.mordvinov@flant.com>
Signed-off-by: Dmitry Mordvinov <dmitry.mordvinov@flant.com>
Signed-off-by: Dmitry Mordvinov <dmitry.mordvinov@flant.com>
…erf#7805) ## Summary Bump `werf/nelm` to `06096861c106` so concurrent release tracking no longer data-races on the progress table printer. The race was reproduced under the race detector in `e2e_complex` during release uninstall, where `ProgressTablesPrinter.Stop` read state that the `Start` worker was still writing. ## What - Deploy no longer triggers a data race in `ProgressTablesPrinter.Start`/`Stop` during concurrent release tracking (werf/nelm#690). - Deploy now matches Kubernetes resource condition types case-insensitively during tracking (werf/nelm feat + werf/kubedog bump to `e8f61a4bc90a`). - The nelm range `8b420d326cbc..06096861c106` also carries werf/nelm#688 (concurrent Helm configuration race); nelm branch `2` was rebased between the two pins, so the range is `diverged` rather than strictly linear. - VERIFIED: `task build` (macOS, non-CGO) succeeds against the new pins. ## Why werf/nelm#690 moves printer state initialization before the tracking goroutine starts, so `Stop` no longer reads a field the worker is concurrently assigning. It is the last dependency race left open in `e2e_complex`. nelm is an ordinary versioned dependency, so the fix lands as a nelm PR plus this bump; the intermediate commits on branch `2` come along with the tip. Signed-off-by: Aleksei Igrychev <aleksei.igrychev@palark.com>
## Summary Enable Go's race detector across the full daily test matrix instead of on every pull request. Per-PR CI keeps race only on `unit` (which already builds with `-race`); all e2e and integration scopes now run against a `-race` werf binary on the daily schedule, where a 1.5–3× slowdown is acceptable. ## What - Per-PR `tests.yml` runs no race-instrumented e2e/integration jobs; `unit` stays `-race` as before. - Every `test_daily.yml` test job runs against a `-race` binary: e2e per-k8s-version (simple/complex/extra), integration_main, integration_git, and the per-k8s / per-container-registry / per-k8s-and-cr matrices. - The `set-up-prebuilt-werf-test-binary` action and the `build-with-coverage` task forward `raceDetectorEnabled`, so daily coverage builds are race-instrumented (`-race` and `-cover` combine in one `go build`; VERIFIED: `go version -m` on the built binary shows `-race=true -cover=true`). - `raceDetectorEnabled` input added to `_test_e2e_per-k8s-version.yml`, `_test_integration_per-k8s-version.yml`, `_test_integration_per-container-registry.yml`, `_test_integration_per-k8s-version-and-container-registry.yml`; it already existed on the regular workflows. - Integration workflow job timeouts raised from 60 to 120 minutes to absorb race overhead. - No product code, CLI, or persisted-data changes; workflow and Taskfile only. ## Why Race coverage found and drove fixes for several concurrency bugs (buildx log pause, Buildah build options, go-containerregistry writer, logboek state, nelm progress printer), but a full `-race` run costs 1.5× on most scopes and ~3× on `e2e_complex` (51m vs 17m). Paying that on every PR slows the feedback loop for no proportional benefit once the known races are fixed; the daily schedule keeps ongoing race coverage without taxing per-PR CI. `unit` is cheap under race and already always instrumented, so it stays on the PR path. Signed-off-by: Aleksei Igrychev <aleksei.igrychev@palark.com>
Conflict resolutions, all favouring the v3 code that removed the feature main instrumented or extended: - synchronization, import metadata, nameless images, virtual merge and the kube namespaces scan are gone from 3, so main's changes to them (opstats in lock_manager, import-metadata cleanup report plumbing and its tests, unnamed-image env report test, VirtualMerge cmd flag, CalculateDependencyImportChecksum, namespaces scan test) are dropped. - the registry-side cleanup report is kept and rewired onto 3's field names (KubernetesNamespaceRestrictionByContext, split meta storage). - pkg/docker/timing.go is dropped: 3's docker/cli returns a moby/moby/client, which exposes no accessor for its *http.Client, so the daemon round-trip timing cannot be installed the way main does. The rest of the opstats instrumentation is kept. - pkg/true_git submodule tests keep 3's ginkgo suite, which supersedes the testify tests main renamed in place. - go.mod/go.sum keep 3's dependency set, which is newer throughout and no longer carries the *-for-werf-helm forks. Signed-off-by: Aleksei Igrychev <aleksei.igrychev@palark.com>
The merge of main into 3 brought two features whose surfaces partly describe subsystems v3 has removed: - the cleanup report's importMetadata item type had no producer left, since 3 replaced import metadata with the source image content digest, yet both language docs still listed it as a value a consumer may see; - opstats declared "import checksum" and "stage lock wait (storage)" operations whose only call sites were the import checksum path and the removed synchronization subsystem. Also state which repository imageMetadata and managedImage were deleted from: on 3 they live in --meta-repo when one is configured, so claiming every non-final item comes from --repo was wrong. Signed-off-by: Aleksei Igrychev <aleksei.igrychev@palark.com>
werf cleanup and werf purge on v3 accept --meta-repo, so imageMetadata and managedImage items may come from a repository the report never named: it carried only repo and finalRepo, and a reader had no way to tell which address a metadata item was deleted from. The report now carries metaRepo, omitted when metadata is not stored apart from the stages. Signed-off-by: Aleksei Igrychev <aleksei.igrychev@palark.com>
Records the ancestry of the content merge landed in werf#7808: the commit there was created after git dropped MERGE_HEAD, so it carried a single parent and left main's six commits looking unmerged to every PR based on 3. The tree is already the reviewed result of that merge, hence -s ours: nothing in the working tree changes. Signed-off-by: Aleksei Igrychev <aleksei.igrychev@palark.com>
Resolving the main-into-3 merge in a detached worktree and creating the branch afterwards dropped MERGE_HEAD, so the pushed "merge" carried one parent: 3 held main's content while every PR based on 3 still saw main's commits as unmerged and reconflicted on the same 19 files. The skill said nothing about either the ordering or the check that catches it. Signed-off-by: Aleksei Igrychev <aleksei.igrychev@palark.com>
…ection The English keep-list use-cases list started with an empty bullet and had no blank line after the intro, which broke rendering; align it with the Russian page. Remove the section on running werf v2 and v3 against one --repo with --meta-repo from both language versions. Signed-off-by: Aleksei Igrychev <aleksei.igrychev@palark.com>
Signed-off-by: Aleksei Igrychev <aleksei.igrychev@palark.com>
Signed-off-by: Aleksei Igrychev <aleksei.igrychev@palark.com> # Conflicts: # .github/actions/set-up-prebuilt-werf-test-binary/action.yml # .github/workflows/_docs_deploy_production.yml # .github/workflows/_test_e2e_per-k8s-version.yml # .github/workflows/_test_e2e_regular.yml # .github/workflows/_test_integration_per-container-registry.yml # .github/workflows/_test_integration_per-k8s-version-and-container-registry.yml # .github/workflows/_test_integration_per-k8s-version.yml # .github/workflows/_test_integration_regular.yml # .github/workflows/_test_unit.yml # .github/workflows/docs_deploy_latest.yml # .github/workflows/release_release-please.yml # .github/workflows/test_daily.yml # CHANGELOG.md # go.mod # go.sum # pkg/build/build_phase.go # pkg/buildah/native_linux.go # pkg/cleaning/cleanup_test.go # pkg/container_backend/docker_server_backend.go
Signed-off-by: Aleksei Igrychev <aleksei.igrychev@palark.com> # Conflicts: # .github/workflows/_test_unit.yml # .release-please-manifest.json # AGENTS.md # CHANGELOG.md # CONTRIBUTING.md # pkg/cleaning/cleanup.go # pkg/cleaning/cleanup_test.go # pkg/config/stapel_image_base_test.go # test/e2e/build/images_dependencies_test.go
Release-As: v3.2.0-dk.1 Signed-off-by: Aleksei Igrychev <aleksei.igrychev@palark.com>
Signed-off-by: Aleksei Igrychev <aleksei.igrychev@palark.com>
An image whose content tag is resolved in BeforeImageStages short-circuits in conveyor.doImage, so AfterImageStages never runs and the image has no last non-empty stage. finalStageDescForImage reached through that stage for the final repo descriptor and dereferenced nil, killing every SBOM build and `werf sbom get` that hit the cache. A single-platform image has no separate final descriptor to begin with: publishFinalImage stores the final repo descriptor in the content tag desc, which convergeImageSbom already passes as the SBOM target, and PropagateArtifacts skips copying when source and destination match. Return nil for that case and keep the multiplatform lookup, which reads the descriptor MultiplatformImage actually tracks. Signed-off-by: Aleksei Igrychev <aleksei.igrychev@palark.com>
The upstream merge reconciled the one-argument and three-argument variants of this helper with a variadic tail, where the first optional writer meant stdout, the second stderr and a third would have been dropped silently. CODESTYLE forbids optional positional arguments, and the single real caller passes all three anyway. Take the three writers explicitly again and pass nil in the test. Signed-off-by: Aleksei Igrychev <aleksei.igrychev@palark.com>
deleteOrphanedArtifacts was covered per storage, but nothing exercised cleanupOrphanedArtifacts, so the claim that a configured final repo is cleaned too — and that a failure names the repo it came from — rested on reading the code. Drive the manager with a fake primary storage and a mocked final one. Signed-off-by: Aleksei Igrychev <aleksei.igrychev@palark.com>
Both merges resolved this file in favour of deleting it, while .agents/skills/git-conventions/SKILL.md points at CONTRIBUTING.md#conventions as the source of truth for commit types and scopes — so the branch shipped a mandatory skill referencing a file that was not there. The restored copy is the delivery-kit one, a strict superset of upstream's with the fork's own scopes (sbom, vex, sign, verify, elf, spec, wiki) that branch 3 commits need. Signed-off-by: Aleksei Igrychev <aleksei.igrychev@palark.com>
e2e_complex needs 47 minutes on branch 3 against a 60 minute limit, and it was cancelled at the limit once the jobs of one run competed for the shared runner. Thirteen minutes of headroom turns the timeout into a recurring false red that says nothing about the code under test. Signed-off-by: Aleksei Igrychev <aleksei.igrychev@palark.com>
Upstream werf selects the release-please manifest by branch (.release-please/<branch>/manifest.json), while the fork keeps a single root .release-please-manifest.json. Every upstream merge therefore drags the upstream manifest of whatever branch it came from into the fork, where it sits next to the live one as dead config advertising a version nobody releases. Adopt the upstream layout. The release-please CLI defaults to the root manifest, so both subcommands now get --manifest-file and --config-file explicitly; without them they would read no manifest at all and treat the repository as never released. The -dk increment and the Release-As override are unchanged. Signed-off-by: Aleksei Igrychev <aleksei.igrychev@palark.com>
…est-3 chore(release): move the manifest per branch, as upstream does
Signed-off-by: Aleksei Igrychev <aleksei.igrychev@palark.com>
…3-v3 chore(release): merge origin/3 and main into delivery-kit/3
chore(3): release 3.2.0-dk.1
Raising timeout-minutes to 90 moved the failure instead of removing it: the job stopped being cancelled at 60 minutes, and e2e_complex then died on ginkgo's own default one-hour suite timeout with `FAIL! - Suite Timeout Elapsed`, two specs marked TIMEDOUT mid-build and half the suites reported as never started. The suite needs 56 minutes on this branch and 31 on main, so pass --timeout explicitly, five minutes under the job limit. The extra suite has also died twice with `Ginkgo timed out waiting for all parallel procs to report back`, which produces no spec-level report at all — the job log then names only the failing suite. Poll progress so a spec that hangs prints its name and goroutines instead. Running the same specs on a dedicated host serially, at four procs and at fifteen procs left them green, so the hang needs the loaded shared runner to show up and the report has to come from there. Signed-off-by: Aleksei Igrychev <aleksei.igrychev@palark.com>
Signed-off-by: Aleksei Igrychev <aleksei.igrychev@palark.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Align delivery-kit with the current werf v3 release line while retaining the fork-specific SBOM, OCI attestation, image signing, and CI behavior. The branch includes
delivery-kit/mainand the latestwerf/werf:3tip available during the merge.What
Version transition
werf/werf@ea66ed8b9.Delivery-kit capabilities
build.sbomand image-level SBOM configuration continue to generate and validate SBOMs, including Go, Python, Rust, Lua, JavaScript, and OS package inputs.werf sbom getpreserves the complete calculated build options when it needs to build an image.--without-imagesskip SBOM convergence instead of dereferencing an empty image graph.headbinary instead of thecatbinary the v3 stapel toolchain dropped.docker, import-metadata, and import-by-file-checksum paths is preserved in runtime and English/Russian template documentation; delivery-kit features use the v3 graph and import-by-source-image-tag model.Why
Delivery-kit had continued on the v2 architecture while werf v3 removed legacy build and storage paths and changed the stage graph, digest, deployment, and validation APIs. A direct merge left fork features connected to removed APIs; porting the integration points preserves delivery-kit behavior without restoring the v2 architecture or dropping current v3 fixes.