Skip to content

pyemscripten/pyodide wheel builds#14787

Open
greateggsgreg wants to merge 34 commits into
pyca:mainfrom
greateggsgreg:pyodide-builds
Open

pyemscripten/pyodide wheel builds#14787
greateggsgreg wants to merge 34 commits into
pyca:mainfrom
greateggsgreg:pyodide-builds

Conversation

@greateggsgreg
Copy link
Copy Markdown
Contributor

@greateggsgreg greateggsgreg commented May 3, 2026

Enable building pyemscripten/pyodide wheels. Now that https://peps.python.org/pep-0783/ is accepted, Pyodide/PyEmscripten wheels (specifically the Pyemscripten ABI (2026_0) ) are allowed to be published on PyPI.

This should also remove the need for pyodide-recipies to maintain a cryptography wheel build script out of tree.

Remaning tasks / discussion in #14788

Enable building pyemscripten/pyodide wheels. Now that https://peps.python.org/pep-0783/ is accepted, Pyodide/PyEmscripten wheels are allowed to be published on PyPI. This enables the last stable release of the Pyodide/Pyemscripten ABI (2025_0), but, will be revved to pyemscripten_2026_0* when it is available.

pyca-boringbot would need to be updated to automatically bump the OpenSSL version. Happy to add that after this.
@alex
Copy link
Copy Markdown
Member

alex commented May 3, 2026

We don't ship wheels for any platform for which we don't run tests, so that's the first step here if you're interested in pursuing this.

@greateggsgreg
Copy link
Copy Markdown
Contributor Author

We don't ship wheels for any platform for which we don't run tests, so that's the first step here if you're interested in pursuing this.

Definitely interested seeing this through. Started #14788 to track discussion and remaining TODOs. Just pushed tests and looking to improve them.

@alex
Copy link
Copy Markdown
Member

alex commented May 4, 2026

To be clear, when we say running the tests, we mean as a part of our CI workflow, on every PR, with acceptable performance.

- uses: dtolnay/rust-toolchain@e97e2d8cc328f1b50210efc529dca0028893a2d9
with:
# Pyodide's emscripten build passes nightly-only `-Z` flags
# (link-native-libraries, emscripten-wasm-eh).
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should be possible to build on stable soon, since neither of these flags are needed any longer. It seems like it might still be passing link-native-libraries=yes somewhere which is the default but it's still nightly only.

Copy link
Copy Markdown
Contributor Author

@greateggsgreg greateggsgreg May 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wrote a TODO near this line and in ci.yml to make sure we validate if we still need nightly by the time pyodide 314.0 is released.

Also updated the issue task list to make sure we don't forget about this.

greateggsgreg and others added 2 commits May 9, 2026 16:27
pypa/cibuildwheel#2812 was merged, so we no longer need the fork pin.
Switch to pypa/cibuildwheel@main until the changes ship in a tagged
release (latest is v3.4.1, which predates the merge).
Comment thread .github/workflows/ci.yml Outdated
Comment thread .github/workflows/ci.yml Outdated
Comment thread pyproject.toml Outdated
Comment thread .github/workflows/ci.yml

- uses: ./.github/actions/upload-coverage

pyemscripten:
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've taken a read through of this, and here is my concern: this is basically a significant amount of configuration, that none of hte pyca/cryptography knows anything about. We don't know when versions should be bumped, don't know what compatibility looks like, don't know how to debug. It duplicates a ton of our other CI code as well.

For this to be a realistic thing we were willing to maintain, we need to get this down much closer to "this is a new entry in the existing linux matrix that shares the vast majority of the code with the other CI workflows"

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the read. Maintainability concerns are fair. Two paths I'd like your call on before cutting code:

Option A — Fold into the linux matrix

New row:

- VERSION: "3.14"
  NOXSESSION: "tests-pyemscripten"
  OPENSSL: {TYPE: "pyemscripten", VERSION: "3.6.2"}
  PYODIDE: {VERSION: "314.0.0a1", EMSDK: "5.0.3"}
  RUST: "nightly"
  RUST_TARGETS: "wasm32-unknown-emscripten"

Pros: checkout/setup-python/rust-toolchain/cache/upload-coverage reuse linux's steps; YAML shrinks; pyemscripten appears as a row in the linux matrix.
Cons: linux job grows 4-5 if: matrix.PYTHON.PYODIDE branches (install + Tests must diverge — nox can't drive a pyodide venv since uv has no wasm build).

Option B — Standalone job, shrunk via shared building blocks

  • New composite action .github/actions/setup-pyodide-build/ for the pyodide-build[resolve] + xbuildenv install + install-emscripten trio.
  • Existing build_openssl.sh (TYPE=pyemscripten already supported).
  • New .github/bin/test_pyemscripten.sh for pyodide build .pyodide venv → install wheel → pytest --cov. Same script runs locally.
  • Local reproducer (Containerfile + driver, already prototyped) so a maintainer can repro CI's pipeline with one command.

Pros: YAML shrinks; no conditional proliferation; unique logic in one action + one script, both grep-able and locally runnable.
Cons: still its own job rather than a row in the linux matrix.

I lean B — fewer conditionals, better local-debug story — but it's your call.

Independent of A vs B, on the maintainability gap: I'd add docs/development/pyemscripten.rst covering the Pyodide ↔ Emscripten ↔ Python compatibility triple (and where Pyodide pins it in Makefile.envs), bump cadence, and 4-5 failure-triage entries. I'll do this regardless of which option we go with.

What do you think?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think one option could be to just use cibuildwheel in all workflow files, as we ensure in our Pyodide platform code that the right Emscripten toolchain is picked up, and we won't need to worry about the pyodide-build version, etc. All dependencies are pinned/constrained, too, making it more reliable and reducing the bespoke configuration involved. The only thing to pick up and update would be based on changes between cibuildwheel releases, i.e., which Pyodide ABIs are available to build and whether to enable or disable them as needed. That said, I'd recommend building only for the 2026 ABI, as the 314.0.0 stable release is coming soon (issue, milestone). The second alpha is out as of two days ago.

First tagged release with cp314-pyodide_wasm32 / pyemscripten_2026_0_wasm32
support (pypa/cibuildwheel#2812). Replaces the git+main pin and unblocks
reproducible wheel builds.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

4 participants