workflows: add build-litellm.yml for riscv64 manywheel builds by luhenry · Pull Request #299 · riseproject-dev/python-wheels

luhenry · 2026-08-21T08:09:55Z

What

Adds .github/workflows/build-litellm.yml to build litellm 1.97.0 riscv64 wheels and publish them to pypi.riseproject.dev.

Why litellm is special

As of the 1.9x line, litellm is no longer pure-Python: its build backend is maturin and it ships a PyO3/Rust extension (litellm.rust_bridge._native, crate at litellm-rust/crates/python-bridge). The pyo3 crate enables abi3-py310, so upstream publishes exactly one cp310-abi3 wheel per arch (loadable on CPython 3.10–3.14) and no free-threaded wheel.

Build shape (sdist → bdist)

python_sdist (ubuntu-latest): builds the sdist itself from the upstream v1.97.0 checkout (never the prebuilt PyPI sdist). The sdist is self-contained and arch-independent — it bundles the committed litellm-rust/Cargo.lock (pinned crates, no floating resolve; upstream's own test-rust.yml builds --locked) and the prebuilt Next.js admin UI under litellm/proxy/_experimental/out/** — so the riscv64 bdist needs no npm/codegen.
build_wheels (ubuntu-24.04-riscv): a single cibuildwheel run selecting cp312/cp313/cp314. It compiles the Rust once (→ cp310-abi3 wheel), reuses that wheel for the other interpreters (find_compatible_wheel → "Skipping build step…"), and still runs the test phase on each — one wheel out, load-tested on three CPythons. cp314t is excluded (no upstream free-threaded wheel; fastuuid and other riscv64 runtime deps have no cp314t wheels yet). Rust toolchain installed via rustup in CIBW_BEFORE_ALL_LINUX; musllinux skipped (rustup ships no riscv64 musl host toolchain).
publish: the shared publish-wheels action (dry-runs off PR branches).

Testing

The bridge loader (litellm/rust_bridge/loader.py) swallows ImportError and returns None, so import litellm alone would pass even with a broken/missing _native. The smoke test therefore exercises the compiled extension directly: asserts native_bridge_available() is True and calls the zero-arg native _native.gil_stats().

Local validation done

yaml.safe_load + actionlint clean (only the expected ubuntu-24.04-riscv unknown-label warning).
Built the abi3 wheel locally on aarch64; confirmed the cp310-abi3 .so loads on CPython 3.12 and the exact CIBW_TEST_COMMAND runs green with the full runtime-dep chain installed.
Verified the full 13-dep runtime closure resolves to riscv64 wheels under --only-binary for cp312/cp313/cp314 (public PyPI + RISE registry).
Confirmed the crypto backend is riscv64-portable: aws-lc-sys 0.43.0 (pinned in Cargo.lock) ships a prebuilt riscv64gc_unknown_linux_gnu_crypto.rs binding at its tag, and ring 0.17.14 builds for riscv64.

🤖 Generated with Claude Code

litellm is no longer pure-Python: since the 1.9x line it builds via maturin and ships a PyO3/Rust extension (litellm.rust_bridge._native). The pyo3 crate enables abi3-py310, so upstream publishes a single cp310-abi3 wheel per arch (loadable on CPython 3.10-3.14) and no free-threaded wheel. Build shape (sdist -> bdist): - python_sdist (ubuntu-latest): build the sdist ourselves from the upstream v1.97.0 checkout. It is self-contained and arch-independent - bundles the committed litellm-rust/Cargo.lock (pinned crates, no floating resolve) and the prebuilt Next.js admin UI under litellm/proxy/_experimental/out/** - so the riscv64 bdist needs no npm/codegen. - build_wheels (ubuntu-24.04-riscv): one cibuildwheel run selecting cp312/cp313/cp314. It compiles the Rust once (-> cp310-abi3 wheel), reuses that wheel for the other interpreters (find_compatible_wheel "Skipping build step"), and still runs the test phase on each - one wheel out, load-tested on three CPythons. cp314t is excluded (no upstream free-threaded wheel; fastuuid and other riscv64 runtime deps have no cp314t wheels). Rust toolchain via rustup in CIBW_BEFORE_ALL_LINUX; musllinux skipped (no riscv64 musl host toolchain). - Smoke test exercises the compiled extension directly: the bridge loader swallows ImportError and returns None, so it asserts native_bridge_available() and calls _native.gil_stats() rather than relying on `import litellm` alone. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

luhenry · 2026-08-21T09:06:58Z

@threexc @justeph could I please get a review? Thanks!

justeph · 2026-08-21T10:15:50Z

+          # litellm builds a single abi3 wheel (pyo3 `abi3-py310`). Selecting three
+          # interpreters does NOT rebuild three times: cibuildwheel compiles the
+          # Rust once (first tag -> cp310-abi3 wheel), then find_compatible_wheel
+          # matches that wheel for the others ("Skipping build step...") while STILL


Indeed, this is true on PyPI but, our build does not produce only a abi3-py310 wheel, but this is not true for our build here:

3.13 and 3.14 wheels are built after 3.12: https://github.com/riseproject-dev/python-wheels/actions/runs/32461915374/job/96710524640#step:4:1000

the dry run publish job would have upload 3 wheels: https://github.com/riseproject-dev/python-wheels/actions/runs/32461915374/job/96722849675#step:2:456

So the build need to be fixed

justeph · 2026-08-21T10:17:04Z

+          # upstream ships no free-threaded wheel, and our fastuuid/other riscv64
+          # runtime deps have no cp314t wheels yet. Only manylinux: rustup.rs has no
+          # riscv64 musl host toolchain (same reason build-fastuuid/tiktoken skip it).
+          CIBW_BUILD: "cp312-manylinux_riscv64 cp313-manylinux_riscv64 cp314-manylinux_riscv64"


nit maybe this is more concise and clear?

Suggested change

CIBW_BUILD: "cp312-manylinux_riscv64 cp313-manylinux_riscv64 cp314-manylinux_riscv64"

CIBW_BUILD: "cp312 cp313 cp314"

CIBW_SKIP: "*-musllinux_*"

justeph · 2026-08-21T10:17:27Z

+          # runtime deps have no cp314t wheels yet. Only manylinux: rustup.rs has no
+          # riscv64 musl host toolchain (same reason build-fastuuid/tiktoken skip it).
+          CIBW_BUILD: "cp312-manylinux_riscv64 cp313-manylinux_riscv64 cp314-manylinux_riscv64"
+          CIBW_BUILD_VERBOSITY: '1'


nitpick: do we need this, or it's only for debugging?

justeph · 2026-08-21T10:26:45Z

+  UV_EXTRA_INDEX_URL: https://pypi.riseproject.dev/simple/
+  UV_INDEX_STRATEGY: unsafe-best-match
+  UV_ONLY_BINARY: ':all:'
+  MANYLINUX_RISCV64_IMAGE: quay.io/pypa/manylinux_2_39_riscv64


any reason to define this? this is the default image for riscv64 selected by cibuildwheel.
One issue I can imagine is that: setting this, will try to use the latest tag available which may not be equivalent to the image selected by cibuildwheel which pin a tag for each cibuildwheel version https://github.com/pypa/cibuildwheel/blob/main/cibuildwheel/resources/pinned_docker_images.cfg#L51. So unless really necessary, I would just drop this and CIBW_MANYLINUX_RISCV64_IMAGE bollow

justeph · 2026-08-21T10:27:01Z

+          # riscv64 musl host toolchain (same reason build-fastuuid/tiktoken skip it).
+          CIBW_BUILD: "cp312-manylinux_riscv64 cp313-manylinux_riscv64 cp314-manylinux_riscv64"
+          CIBW_BUILD_VERBOSITY: '1'
+          CIBW_MANYLINUX_RISCV64_IMAGE: ${{ env.MANYLINUX_RISCV64_IMAGE }}


drop this, see previous comment

justeph · 2026-08-21T10:28:50Z

+          tar zxf "${{ steps.fetch_sdist.outputs.download-path }}/${{ needs.python_sdist.outputs.sdist_artifact_name }}" \
+            --strip-components=1 -C litellm
+          uv pip install --upgrade cibuildwheel
+          python -m cibuildwheel --output-dir wheelhouse ./litellm


why not using the official pypa/cibuildwheel actions? we could also avoid the uv install step before

justeph · 2026-08-21T10:33:02Z

+            UV_EXTRA_INDEX_URL=https://pypi.riseproject.dev/simple/
+            UV_INDEX_STRATEGY=unsafe-best-match


do we need the UV variable? I think cibuildwheel uses venv + pip and not uv unless we specifically ask it

justeph · 2026-08-21T10:35:00Z

+        with:
+          name: ${{ needs.python_sdist.outputs.sdist_artifact_name }}
+
+      - name: Install uv


I think we can skip this one, see comment below

Git tag has features=["extension-module"] (no abi3); upstream adds abi3-py310 only at release time. Building from source produced 3 per-interpreter wheels + 3x Rust compile. Set abi3-py312 on the sdist so maturin emits one cp312-abi3 wheel, deduped across cp312/313/314. - use pypa/cibuildwheel action with sdist as package-dir; drop manual tar + uv install - drop MANYLINUX_RISCV64_IMAGE override (cibuildwheel pins by digest) - CIBW_BUILD "cp312 cp313 cp314" + CIBW_SKIP musllinux - drop UV_* env and CIBW_BUILD_VERBOSITY Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Co-authored-by: Julien Stephan <jstephan@baylibre.com>

luhenry requested review from justeph and threexc August 21, 2026 09:06

justeph requested changes Aug 21, 2026

View reviewed changes

justeph reviewed Aug 21, 2026

View reviewed changes

Comment thread .github/workflows/build-litellm.yml Outdated

justeph mentioned this pull request Aug 21, 2026

workflows: add build-torchaudio.yml for riscv64 wheels #285

Open

Update .github/workflows/build-litellm.yml

07a8740

Co-authored-by: Julien Stephan <jstephan@baylibre.com>

Navigation Menu

Uh oh!

workflows: add build-litellm.yml for riscv64 manywheel builds - #299

workflows: add build-litellm.yml for riscv64 manywheel builds#299
luhenry wants to merge 3 commits into
mainfrom
litellm

luhenry commented Aug 21, 2026

Uh oh!

luhenry commented Aug 21, 2026

Uh oh!

Uh oh!

justeph Aug 21, 2026

Uh oh!

justeph Aug 21, 2026

Uh oh!

justeph Aug 21, 2026

Uh oh!

justeph Aug 21, 2026

Uh oh!

justeph Aug 21, 2026

Uh oh!

justeph Aug 21, 2026

Uh oh!

justeph Aug 21, 2026

Uh oh!

justeph Aug 21, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	CIBW_BUILD: "cp312-manylinux_riscv64 cp313-manylinux_riscv64 cp314-manylinux_riscv64"
	CIBW_BUILD: "cp312 cp313 cp314"
	CIBW_SKIP: "-musllinux_"

		UV_EXTRA_INDEX_URL=https://pypi.riseproject.dev/simple/
		UV_INDEX_STRATEGY=unsafe-best-match

Uh oh!

Conversation

luhenry commented Aug 21, 2026

What

Why litellm is special

Build shape (sdist → bdist)

Testing

Local validation done

Uh oh!

luhenry commented Aug 21, 2026

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants