Skip to content

Parallelize WASM extension builds#399

Merged
adamziel merged 4 commits intotrunkfrom
codex/parallelize-wasm-extension-builds
May 5, 2026
Merged

Parallelize WASM extension builds#399
adamziel merged 4 commits intotrunkfrom
codex/parallelize-wasm-extension-builds

Conversation

@adamziel
Copy link
Copy Markdown
Collaborator

@adamziel adamziel commented May 4, 2026

What it does

Runs the WASM extension PHP-version matrix concurrently and shares the expensive Playground base image across the matrix. Both the PR smoke workflow and the publish workflow now build playground-php-wasm:base once, upload it as a short-lived internal artifact, then fan out all six PHP 8.x builds in parallel.

Rationale

The publish workflow was running one PHP build at a time. In the observed trunk run, PHP 8.5 finished before PHP 8.4 started, and the build spent about 24m30s inside build-in-docker-rust.sh for that one PHP version.

Blindly running all six jobs in parallel exposed another bottleneck: every job rebuilt the same Ubuntu-based Playground base image and hit apt-get update at the same time. Sharing that base image keeps the wall-time win while avoiding six duplicate apt/install passes.

A later CI run showed one more duplicate path: @php-wasm/compile-extension@3.1.27 rebuilt the Playground base image and PHP-specific compile-extension image again during Stage 2. The build script now reuses the images prepared in Stage 0, so each matrix job does one Docker image preparation pass instead of two.

Implementation

  • Add a base-image job to .github/workflows/wasm-spike.yml and .github/workflows/publish-wasm-extension-artifact.yml
  • Upload the saved playground-php-wasm:base image with retention-days: 1
  • Make each PHP matrix job download and docker load that base image before building its PHP-specific compile-extension image
  • Set max-parallel: 6 for both PHP matrices
  • Add SKIP_BASE_IMAGE_BUILD=1 support to build-in-docker-rust.sh so CI can require a preloaded base image while local runs still build it normally
  • Patch the pinned compile-extension CLI at install time to skip its duplicate Docker image build and use the Stage 0 images; the patch fails loudly if the package shape changes

I also tried wiring docker buildx build --cache-to type=gha, but dropped it. The GHA cache backend requires a non-default Buildx driver, and that driver does not automatically expose locally tagged intermediate images like playground-php-wasm:base to later Dockerfile FROM lines. The internal image artifact is simpler and matches this build's local image chain.

Follow-up: @php-wasm/compile-extension should expose a first-class way to use already-prepared images so this repo does not need to patch the pinned CLI at runtime.

Testing instructions

  • bash -n packages/php-ext-wp-mysql-parser/wasm-spike/build-in-docker-rust.sh
  • actionlint .github/workflows/publish-wasm-extension-artifact.yml .github/workflows/wasm-spike.yml
  • git diff --check
  • Verified in PR CI run 25348528195: shared base image job passed in 4m18s, all six PHP matrix jobs passed in 9m30s-11m55s, and the workflow completed successfully

@adamziel adamziel marked this pull request as ready for review May 5, 2026 06:27
@adamziel adamziel merged commit 8c3c3dd into trunk May 5, 2026
31 checks passed
@adamziel adamziel deleted the codex/parallelize-wasm-extension-builds branch May 5, 2026 06:27
adamziel added a commit that referenced this pull request May 5, 2026
## What changed

- Raises the WASM extension matrix build timeout from 45 to 60 minutes
in the publish workflow.
- Keeps the same timeout in the interactive WASM spike workflow, which
runs the same build shape.

## Why

The first publish run after merging #399 built the PHP 8.3 side module
successfully, but hit the 45-minute job timeout before the verify/upload
steps could run. The log ended with the artifact written and then the
job cancellation.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant