[Packaging] Make generated binary and WASM builds reproducible and non-mutating

[cssbruno]

Priority

Medium-high — reproducible packaging, module safety, and trustworthy artifact hashes

Context

Generated application packaging currently performs these steps in internal/appgen/build.go:

tidyGeneratedApp(absApp, goEnv) // go mod tidy
exec.Command("go", "build", "-buildvcs=false", "-o", absBinary, "./cmd/server")

The WASM path follows the same pattern with GOOS=js and GOARCH=wasm.

The build environment inherits most of os.Environ, disables GOWORK, and does not currently pass -trimpath or define a broader reproducibility contract.

This behavior predates #663, which proposes generating applications as packages inside the original application module instead of maintaining a nested gowdk-generated-app module. Continuing to run go mod tidy as an implicit packaging step becomes more dangerous once generated code shares the application module context.

Problem

Packaging mutates module state

go mod tidy is a dependency-management operation, not a read-only compilation prerequisite. It may rewrite go.mod and go.sum, remove requirements, add requirements, or resolve modules according to the active toolchain and environment.

Today that mutates the generated module. Under the direction in #663 it could mutate the user's application module unless the build flow is changed first.

A command whose primary contract is “compile this generated app” should not silently edit module metadata.

Absolute paths can affect artifacts

Without -trimpath, generated binaries can retain build-system paths in symbol/debug metadata. Identical projects built from different checkout directories may therefore produce different bytes and disclose local build paths.

The effective build environment is implicit

Inherited values such as these can change output or dependency resolution:

• GOFLAGS;
• GOOS / GOARCH;
• CGO_ENABLED;
• compiler/linker variables for CGO;
• build tags;
• GOWORK;
• module proxy/private-module settings;
• selected Go toolchain.

Some variation is legitimate, but it should be explicit in the packaging contract and recorded in build metadata rather than silently undermining reproducibility claims.

Reproducibility is not tested end to end

Generated source/output determinism tests do not prove that the final native binary or WASM artifact is identical when built from equivalent source snapshots in different absolute directories.

Goal

Make binary and WASM packaging read-only with respect to application module files and define a testable reproducibility envelope for identical inputs, toolchain, target tuple, and declared build settings.

Required behavior

1. Remove implicit go mod tidy from packaging

BuildBinary and BuildWASM must not modify the source or generated application module as part of compilation.

Preferred options include:

• generate complete deterministic module metadata before packaging when a nested/staged module still exists;
• validate module consistency with read-only Go commands;
• build with -mod=readonly where applicable;
• use a temporary staging workspace/module if dependency preparation is necessary;
• run go mod tidy -diff only as a diagnostic check when the supported Go toolchain provides it, without applying the diff.

A missing or inconsistent dependency declaration should produce an actionable error showing the read-only remediation, not silently rewrite files.

After #663, packaging must never run go mod tidy against the user's root module.

2. Use path-independent compiler flags

Native and WASM builds should use at least:

-trimpath
-buildvcs=false

and the selected read-only module policy.

Evaluate Go build-ID behavior before adding linker flags such as -buildid=. Do not remove useful integrity/cache metadata without evidence. The reproducibility contract should be based on measured output and supported Go behavior, not assumptions.

3. Define the reproducibility envelope

Document precisely when byte-for-byte equality is expected. A reasonable first contract is:

Identical validated application plan, generated source, dependencies, Go toolchain, target GOOS/GOARCH, CGO/toolchain settings, build tags, and deterministic GOWDK packaging options produce identical artifact bytes regardless of checkout/output absolute path.

Do not claim equality across:

• different Go toolchain versions;
• different target operating systems or architectures;
• uncontrolled CGO compiler/linker versions;
• application code that intentionally embeds nondeterministic data;
• differing build tags or linker flags.

For the primary release path, prefer a well-defined CGO_ENABLED=0 reproducible lane where application dependencies permit it.

4. Normalize or record output-affecting environment

Create a typed packaging environment rather than inheriting every variable without classification.

At minimum:

• set target values explicitly;
• set GOWORK according to the documented module strategy;
• decide whether GOFLAGS is cleared, validated, or included in the build identity;
• record Go version/toolchain, target, CGO, tags, and effective deterministic flags;
• keep secrets and proxy credentials out of reports;
• do not include absolute work/cache paths in the semantic build identity.

A future strict flag may be useful:

gowdk build --reproducible

but non-mutating module behavior and -trimpath should not require opt-in. Decide whether strict environment rejection is default or flag-controlled based on compatibility.

5. Integrate deterministic build identity

Use #687's plan/build identity as deterministic generated metadata rather than injecting timestamps or uncontrolled VCS state.

If the binary embeds version/build metadata, it should come from explicit deterministic inputs such as:

• GOWDK version;
• plan/build ID;
• target name and role;
• schema/protocol versions.

Do not inject current time, temporary directories, process IDs, or hostnames into release artifacts.

6. Publish final files atomically

Compile native/WASM output to a temporary sibling path, validate it, then atomically replace the final destination according to #669's publication model.

A failed build must not truncate or replace the previous valid binary/WASM artifact.

7. Expose a verification path

Build reports/timings should record safe packaging metadata:

{
  "goVersion": "go1.x.y",
  "goos": "linux",
  "goarch": "amd64",
  "cgoEnabled": false,
  "trimpath": true,
  "moduleMode": "readonly",
  "artifactSha256": "..."
}

Do not record secret environment values or private proxy credentials.

A future gowdk verify command may compare the artifact hash with the committed build identity/manifest.

Module strategy details

Before #663

If generated apps retain a compiler-owned nested module temporarily:

• module files must be generated deterministically;
• dependency validation must be read-only during packaging;
• any required normalization should happen in a staging directory before transactional publication;
• repeated packaging should leave generated go.mod/go.sum unchanged.

After #663

When generated apps are ordinary packages in the application module:

• no nested go.mod should be created;
• no packaging command should edit the application go.mod or go.sum;
• application replace, exclude, vendoring, and workspace policy should be honored according to the documented build mode;
• generated package imports should have been validated before invoking go build.

Test plan

Non-mutation tests

• record hashes/bytes of application go.mod and go.sum before native build;
• build successfully and assert they are unchanged;
• repeat for WASM;
• repeat for a failed build;
• assert git status --porcelain is unchanged after packaging in a clean fixture;
• test current nested-module behavior and the Generate apps inside the project module instead of a nested gowdk-generated-app module #663 module-local direction;
• verify generated module files remain unchanged across repeated builds.

Reproducibility tests

Materialize the same fixture in two different absolute root paths and build with the same pinned toolchain/settings:

• compare native binary SHA-256;
• compare WASM SHA-256;
• inspect binaries for leaked checkout paths;
• repeat with spaces and non-ASCII checkout paths;
• repeat after clearing caches to avoid a false cache-only result;
• confirm changing a semantic input changes the hash;
• confirm changing only the absolute checkout path does not;
• confirm changing Go version/target/settings is reported as outside or different within the identity envelope.

Use go version -m or equivalent inspection to verify intended module/build settings without asserting unstable formatting unnecessarily.

Failure/publication tests

• previous valid binary remains intact after compile failure;
• previous valid WASM remains intact after compile failure;
• temporary output is cleaned after success, failure, and cancellation;
• cross-device/unsupported atomic replacement behavior is diagnosed clearly;
• Windows executable replacement semantics receive focused coverage.

Environment tests

• inherited GOFLAGS behavior is explicit and tested;
• GOWORK behavior matches documentation;
• CGO settings appear in safe build metadata;
• secret-like environment variables never appear in reports;
• vendor/read-only module mode works where configured.

Acceptance criteria

• Native and WASM packaging never run a mutating module command against the application module.
• go.mod and go.sum remain byte-for-byte unchanged after successful and failed packaging.
• Packaging uses -trimpath and disables uncontrolled VCS stamping.
• Module resolution is read-only or occurs only in an isolated staging area before transactional publication.
• The reproducibility envelope is documented with explicit toolchain, target, CGO, tag, and environment boundaries.
• Equivalent fixtures in different absolute directories produce identical native/WASM hashes within the supported reproducible lane.
• Checkout, temp, and cache paths are not embedded in reproducible artifacts.
• Safe packaging metadata and final SHA-256 values are available in build reports/identity data.
• Failed packaging preserves the previous valid artifact.
• Behavior remains compatible with local replace, vendoring, private modules, and the module strategy selected by Generate apps inside the project module instead of a nested gowdk-generated-app module #663.
• Unix, macOS, and Windows packaging paths receive focused tests.

Non-goals

• Guaranteeing identical native bytes across different Go versions, operating systems, architectures, or uncontrolled CGO toolchains.
• Replacing Go module integrity (go.sum) or release attestations.
• Downloading all dependencies into the repository.
• Embedding timestamps for cosmetic build information.
• Solving general reproducibility of arbitrary user-owned external tools.

Related

• [Supply Chain] Pin privileged workflow inputs and harden installer bootstrap #640 — deterministic privileged release inputs and installer hardening
• Generate apps inside the project module instead of a nested gowdk-generated-app module #663 — generate apps inside the application module
• Publish build and generated-app output transactionally #669 — transactional artifact publication
• [Compiler] Introduce one bounded subprocess supervisor for all external tools #685 — bounded subprocess supervisor
• [Artifacts] Embed a deterministic build identity across all generated outputs #687 — deterministic build identity
• [Build] Validate the complete artifact output topology before writing #694 — artifact destination topology validation