Skip to content

workflows: add build-protobuf.yml for riscv64 manywheel builds - #280

Merged
luhenry merged 1 commit into
mainfrom
protobuf
Aug 20, 2026
Merged

workflows: add build-protobuf.yml for riscv64 manywheel builds#280
luhenry merged 1 commit into
mainfrom
protobuf

Conversation

@luhenry

@luhenry luhenry commented Aug 20, 2026

Copy link
Copy Markdown
Member

What

Adds .github/workflows/build-protobuf.yml to build and publish protobuf riscv64 wheels to pypi.riseproject.dev, following the sdist → bdist structure of build-cffi.yml.

Relates to #83

Why protobuf differs from cffi

protobuf's setup.py cannot build from the git checkout — upstream's python README is explicit: "You cannot build from setup.py using the GitHub repo or the GitHub source tarball." The self-contained sdist (bundling the upb/utf8_range C sources and generated *_pb2.py files) is assembled by Bazel via //python/dist:source_wheel.

Jobs

  • python_sdist — Bazel-builds the sdist once on x86. The sdist is architecture-independent ("always the same and does not depend on a local version of Python"), so there's no reason to run Bazel/protoc on riscv64. bazelisk is installed and pinned to Bazel 8.5.2.
  • linux — builds the riscv64 bdist from that sdist with cibuildwheel (matrix: cp312/cp313/cp314/cp314t). The bundled C sources compile into google._upb._message with no protoc, Bazel, or external native dependency — so no CIBW_BEFORE_BUILD. The test asserts the upb backend is active (not a silent fall-back to pure-Python) and round-trips a well-known type.
  • publish — hands off to the shared publish-wheels action, which also auto-creates docs/packages/protobuf.yaml.

Versioning

The workflow input is the git tag (e.g. 35.1); the Python package version (7.35.1) is derived from the built sdist filename, since protobuf's tag scheme (vNN.M) differs from its Python version (7.NN.M).

Notes

  • Validated locally: building a wheel from the official 7.35.1 sdist via pip wheel produces a working upb extension.
  • The commit is unsigned (the signing environment was unavailable when it was created).

🤖 Generated with Claude Code

Comment thread .github/workflows/build-protobuf.yml Outdated
@luhenry

luhenry commented Aug 20, 2026

Copy link
Copy Markdown
Member Author

@threexc @justeph its ready for review

@justeph

justeph commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

@luhenry since the only merge option is "squash and merge" I would prefer you to open a new PR with all the claude related changes, otherwise it will be squashed into the workflows: add build-protobuf.yml for riscv64 manywheel builds commit which is really weird

@luhenry

luhenry commented Aug 20, 2026

Copy link
Copy Markdown
Member Author

@luhenry since the only merge option is "squash and merge" I would prefer you to open a new PR with all the claude related changes, otherwise it will be squashed into the workflows: add build-protobuf.yml for riscv64 manywheel builds commit which is really weird

@justeph done.

@justeph

justeph commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

@luhenry since the only merge option is "squash and merge" I would prefer you to open a new PR with all the claude related changes, otherwise it will be squashed into the workflows: add build-protobuf.yml for riscv64 manywheel builds commit which is really weird

@justeph done.

thank you! I'll do a proper review later today for the protobuf pipeline

@threexc threexc left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Glad to see opportunities to offload some parts of the build to the default runners.

As an aside, Mark originally said this on our protobuf issue in wheel_builder:

We may not want to build this straight away.  There is a pure Python binary available, and building the wheels requires bazel.  It's not clear at the time of writing whether bazel supports riscv64.

@luhenry

luhenry commented Aug 20, 2026

Copy link
Copy Markdown
Member Author

Bazel is only used to build the sdist on x86, then the bdist is built from the sdist (without bazel) on riscv64. It's documented upstream. The pure python version doesn't use the faster native library; see https://github.com/protocolbuffers/protobuf/tree/main/python#implementation-backends.

Comment thread .github/workflows/build-protobuf.yml Outdated
Comment on lines +4 to +5
name: Build protobuf wheels (riscv64)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you add a link to the original file on protobuf repo? like you did for numpy? it's easier for review

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://github.com/protocolbuffers/protobuf doesn't make releases on GitHub, or not that I could find.

protobuf's setup.py cannot build from the git checkout (upstream's python
README is explicit about this); the self-contained sdist that bundles the
upb/utf8_range C sources and generated *_pb2.py files is assembled by Bazel
via //python/dist:source_wheel. Mirror build-cffi.yml's sdist -> bdist split:

- python_sdist: Bazel-build the architecture-independent sdist once on x86.
- linux: build the riscv64 bdist from that sdist with cibuildwheel. The
  bundled C sources compile into google._upb._message with no protoc, Bazel,
  or external native dependency, so no CIBW_BEFORE_BUILD is needed. The test
  asserts the upb backend is active and round-trips a well-known type.
- publish: hand off to the shared publish-wheels action.

The workflow input is the git tag (e.g. 35.1); the Python package version
(7.35.1) is derived from the built sdist filename.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@luhenry
luhenry merged commit 6eb52b4 into main Aug 20, 2026
9 checks passed
@luhenry luhenry linked an issue Aug 24, 2026 that may be closed by this pull request
@luhenry luhenry removed a link to an issue Aug 24, 2026
@luhenry luhenry linked an issue Aug 24, 2026 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

protobuf riscv64 support

3 participants