refactor: 🧹 use hwloc instead of custom topology discovery - #208
Open
robertodr wants to merge 4 commits into
Open
refactor: 🧹 use hwloc instead of custom topology discovery#208robertodr wants to merge 4 commits into
robertodr wants to merge 4 commits into
Conversation
robertodr
requested review from
diagonal-hamiltonian,
fpietra and
ludmilaasb
as code owners
August 5, 2026 12:34
robertodr
changed the title
|
Docs preview: https://pr-208.monoprop-docs.pages.dev |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## refactor-split-impls #208 +/- ##
=====================================================
Coverage 97.58% 97.58%
=====================================================
Files 14 14
Lines 745 745
Branches 99 99
=====================================================
Hits 727 727
Misses 13 13
Partials 5 5
Flags with carried forward coverage won't be shown. Click here to find out more. |
Contributor
There was a problem hiding this comment.
Pull request overview
🤖 AI text below 🤖
Refactors the CPU-topology discovery and partition thread pinning in monoprop to use hwloc, aiming for more portable and consistent topology enumeration and affinity binding across supported platforms.
Changes:
- Replace the custom
/sys-parsing topology logic with anhwloc-backed implementation (CpuTopology.*) and expose a deterministic placement-policy helper for unit testing. - Integrate
hwlocinto the C++ build (CMake + pkg-config) and update C++ unit tests accordingly. - Update developer tooling and CI environments to install
hwloc(scripts, workflows, devcontainer) and refresh related documentation.
Reviewed changes
Copilot reviewed 14 out of 14 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tools/install-deps.sh | Adds optional hwloc installation and broadens OS detection for the dependency installer script. |
| pyproject.toml | Updates macOS cibuildwheel pre-build dependencies to include hwloc. |
| docs/content/docs/features/parallelism.mdx | Updates documentation to reflect hwloc-based pinning support beyond Linux. |
| cpp/tests/cpu_topology_tests.cpp | Reworks CPU-topology tests for hwloc and adds deterministic placement-policy unit tests. |
| cpp/tests/CMakeLists.txt | Links unit tests against hwloc via PkgConfig::HWLOC. |
| cpp/monoprop/detail/partition/CpuTopology.h | Updates the public topology/pinning API docs and types for an hwloc implementation. |
| cpp/monoprop/detail/partition/CpuTopology.cpp | Implements topology discovery and thread pinning via hwloc, plus shared placement-policy logic. |
| cpp/monoprop/CMakeLists.txt | Adds pkg-config discovery for hwloc and wires PkgConfig::HWLOC into library linkage. |
| AGENTS.md | Documents hwloc as a required system dependency and where it’s used. |
| .github/workflows/test.yml | Ensures CI runners install hwloc on macOS and Ubuntu. |
| .github/workflows/qa-analysis.yml | Adds libhwloc-dev to QA workflow dependency installs. |
| .github/workflows/docpages.yml | Adds libhwloc-dev to docs build workflow dependency installs. |
| .github/workflows/copilot-setup-steps.yml | Adds libhwloc-dev to Copilot setup steps. |
| .devcontainer/Dockerfile | Installs libhwloc-dev in the devcontainer image. |
Suppressed comments (1)
tools/install-deps.sh:217
- The script advises setting only
CMAKE_PREFIX_PATH, but the build now discovers hwloc viapkg-config(pkg_check_modules(HWLOC …)), which typically requiresPKG_CONFIG_PATHto include the install prefix when using a non-systemINSTALL_PREFIX. Without that, a custom-prefix hwloc installed by this script may still not be found by CMake.
echo "Dependencies installation completed successfully!"
echo "Install location: $INSTALL_PREFIX"
echo "Make sure to set CMAKE_PREFIX_PATH=$INSTALL_PREFIX when building monoprop"
Comment on lines
105
to
+109
| INTERFACE | ||
| Boost::boost | ||
| Threads::Threads | ||
| $<$<TARGET_EXISTS:MPI::MPI_CXX>:MPI::MPI_CXX> | ||
| PRIVATE |
Comment on lines
83
to
+86
| const auto rank0 = partition::partition_cpusets(/*n=*/1, /*group_index=*/0, /*group_count=*/2); | ||
| const auto rank1 = partition::partition_cpusets(/*n=*/1, /*group_index=*/1, /*group_count=*/2); | ||
| // Off Linux there is no pinning, so both come back empty (unpinned, still disjoint by the scheduler). | ||
| #if defined(__linux__) | ||
| BOOST_CHECK_EQUAL(rank0.size(), 1u); | ||
| BOOST_CHECK_EQUAL(rank1.size(), 1u); | ||
| #else | ||
| BOOST_CHECK(rank0.empty()); | ||
| BOOST_CHECK(rank1.empty()); | ||
| #endif | ||
| BOOST_REQUIRE_EQUAL(rank0.size(), 1u); | ||
| BOOST_REQUIRE_EQUAL(rank1.size(), 1u); |
robertodr
marked this pull request as draft
August 5, 2026 13:27
This comment was marked as resolved.
This comment was marked as resolved.
robertodr
force-pushed
the
refactor-use-hwloc
branch
from
August 5, 2026 14:10
842359c to
f810a4e
Compare
robertodr
force-pushed
the
refactor-use-hwloc
branch
from
August 5, 2026 14:42
f810a4e to
7a8322a
Compare
Panadestein
force-pushed
the
refactor-use-hwloc
branch
from
August 5, 2026 15:45
7a8322a to
d3c3bee
Compare
robertodr
force-pushed
the
refactor-use-hwloc
branch
2 times, most recently
from
August 5, 2026 17:51
aeda5c8 to
8a20dde
Compare
robertodr
force-pushed
the
refactor-use-hwloc
branch
from
August 6, 2026 09:18
8a20dde to
746e541
Compare
robertodr
force-pushed
the
refactor-use-hwloc
branch
from
August 6, 2026 09:20
746e541 to
03ce849
Compare
github-actions
Bot
added
documentation
robertodr
force-pushed
the
refactor-use-hwloc
branch
from
August 6, 2026 10:04
03ce849 to
4344a7c
Compare
robertodr
force-pushed
the
refactor-use-hwloc
branch
from
August 6, 2026 11:47
4344a7c to
b400bce
Compare
robertodr
force-pushed
the
refactor-use-hwloc
branch
from
August 6, 2026 11:48
b400bce to
6ddac83
Compare
robertodr
marked this pull request as ready for review
August 6, 2026 14:18
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 15 out of 15 changed files in this pull request and generated 1 comment.
Suppressed comments (1)
docs/content/docs/features/parallelism.mdx:32
- 🤖 AI text below 🤖
The macOS claim is incorrect. hwloc can discover Darwin topology, but its own binding implementation states that Darwin has no CPU-binding facilities, so hwloc_set_cpubind() here fails and the worker remains unpinned. Document macOS as topology-discovery-only rather than presenting thread pinning as supported.
| `monoprop_PARTITION_PINNING` | `on` | `0`/`false`/`no` disables pinning each partition to a core. Supported on platforms where hwloc can bind threads (Linux, macOS). |
robertodr
commented
Aug 7, 2026
Comment on lines
+17
to
+21
| #include <hwloc.h> | ||
|
|
||
| #include <pthread.h> | ||
| #include <algorithm> | ||
| #include <map> | ||
| #include <vector> |
Member
Author
There was a problem hiding this comment.
Suggested change
| #include <hwloc.h> | |
| #include <pthread.h> | |
| #include <algorithm> | |
| #include <map> | |
| #include <vector> | |
| #include <algorithm> | |
| #include <map> | |
| #include <vector> | |
| #include <hwloc.h> |
Assisted-by: GitHub Copilot: gpt-5.6-sol (plan), claude-haiku-4.5/claude-sonnet-4.6 (execute)
with before-build we re-ran the same installation script before every build.
robertodr
force-pushed
the
refactor-use-hwloc
branch
from
August 7, 2026 08:17
500a915 to
ed369bf
Compare
|
Member
Author
|
Running HeisenbergTime
Memory (peak RSS)
SchrödingerTime
Memory (peak RSS)
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The CPU topology discovery and thread pinning used a custom implementation that was not really portable.
hwlocis the standard tool for this job.Changes
hwlocas dependency for cross-platform CPU topology discovery.Checklist
docs/,CONTRIBUTING.md) if neededCHANGELOG/ release notes updated if applicableAI/LLM disclosure
Important
By opening this PR I confirm that I have read CONTRIBUTING.md and I agree to the terms of the Contributor License Agreement.
Warning
If you're contributing on behalf of your employer, contact cla@algorithmiq.fi to arrange a Corporate CLA.