Skip to content

Migrate RMM usage to CCCL MR design#98

Open
bdice wants to merge 8 commits intoNVIDIA:mainfrom
bdice:rmm-cccl-migration
Open

Migrate RMM usage to CCCL MR design#98
bdice wants to merge 8 commits intoNVIDIA:mainfrom
bdice:rmm-cccl-migration

Conversation

@bdice
Copy link
Copy Markdown

@bdice bdice commented Apr 2, 2026

Summary

  • Migrate all cuCascade memory resource classes from rmm::mr::device_memory_resource virtual inheritance to the CCCL-native resource concept (allocate/deallocate with cuda::stream_ref, allocate_sync/deallocate_sync, operator==, get_property).
  • Replace unique_ptr<device_memory_resource> owning storage with cuda::mr::any_resource<cuda::mr::device_accessible>.
  • Update memory_space::get_default_allocator() to return device_async_resource_ref from the any_resource.
  • Update pool_memory_resource usage (no longer a template in RMM 26.06; upstream passed as device_async_resource_ref).
  • Move memory_reservation.hpp template bodies (get_memory_resource_as<T>(), get_memory_resource_of<Tier>()) to memory_space.hpp to avoid incomplete-type errors.
  • Remove allocator null-checks in GPU and HOST memory_space constructors — any_resource is a value type and cannot be null.
  • Update internal converter helpers in representation_converter.cpp to accept rmm::device_async_resource_ref instead of raw rmm::mr::device_memory_resource*.
  • Update test utilities (shared_device_resource, logging_device_resource, make_mock_memory_space) for the new resource concept.
  • Pre-commit formatting pass on files touched by the merge with upstream/main.

Affected classes

  • null_device_memory_resource
  • numa_region_pinned_host_allocator
  • fixed_size_host_memory_resource
  • reservation_aware_resource_adaptor
  • small_pinned_host_memory_resource
  • memory_space

Notes

  • Companion to the rapidsmpf migration PR.
  • cuda::stream_ref{cudaStream_t{nullptr}} in allocate_sync/deallocate_sync implementations may produce deprecation warnings. These do not break the build because the WARNINGS_AS_ERRORS CMake guard on line 80 checks the wrong variable name (WARNINGS_AS_ERRORS vs the option CUCASCADE_WARNINGS_AS_ERRORS), so -Werror is currently not enabled.
  • Pre-commit reformatted several files that were untouched by the migration but modified by the upstream merge (cuda_utils.hpp, test_data_batch.cpp, test_data_representation.cpp, benchmark_representation_converter.cpp, data_batch.hpp). These are whitespace/include-order changes only.

… resource concept

RMM 26.06 removes rmm::mr::device_memory_resource and adopts CCCL-native
memory resource concepts. This migrates all cuCascade resource classes:

- null_device_memory_resource: replaced virtual override with CCCL concept
  methods (allocate/deallocate with cuda::stream_ref, allocate_sync,
  deallocate_sync, operator==, get_property).
- numa_region_pinned_host_allocator: same pattern.
- fixed_size_host_memory_resource: same pattern.
- reservation_aware_resource_adaptor: same pattern.
- small_pinned_host_memory_resource: same pattern.
- memory_space: unique_ptr<device_memory_resource> replaced with
  cuda::mr::any_resource<cuda::mr::device_accessible> for owning storage;
  get_default_allocator returns device_async_resource_ref from any_resource.
- memory_reservation.hpp: moved template bodies out-of-line to avoid
  incomplete-type errors with memory_space forward declaration.
- common.hpp/cpp: updated factory typedefs and pool_memory_resource usage
  (no longer a template in 26.06).
- pool_memory_resource upstream passed as device_async_resource_ref.
- Test utilities updated for new resource concept.
@copy-pr-bot
Copy link
Copy Markdown

copy-pr-bot bot commented Apr 2, 2026

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@mbrobbel
Copy link
Copy Markdown
Member

mbrobbel commented Apr 7, 2026

/ok to test a852c20

# Conflicts:
#	src/data/representation_converter.cpp
#	src/memory/memory_space.cpp
#	test/utils/cudf_test_utils.cpp
@mbrobbel
Copy link
Copy Markdown
Member

/ok to test 9df76bf

bdice added 2 commits April 16, 2026 08:30
The CCCL async_resource concept requires explicit alignment arguments
and explicit cudaStream_t construction for stream_ref, unlike the old
rmm::mr::device_memory_resource pointer API which had defaults.

- Pass rmm::CUDA_ALLOCATION_ALIGNMENT for device resource allocate/deallocate
- Pass alignof(std::max_align_t) for host resource allocate/deallocate
- Use cuda::stream_ref{cudaStream_t{nullptr}} for sync allocate/deallocate
- Use rmm::device_async_resource_ref in representation_converter signatures
- Construct null_device_memory_resource directly (no unique_ptr indirection)
@mbrobbel
Copy link
Copy Markdown
Member

/ok to test fd00654

@bdice bdice marked this pull request as ready for review April 16, 2026 13:55
bdice added 4 commits April 16, 2026 10:15
…compliance

Test resource types (shared_device_resource, logging_device_resource) now
implement the synchronous allocation interface required by the CCCL
resource concept. Also migrates deprecated set_current_device_resource_ref
calls to set_current_device_resource and fixes any_resource construction
in test setup.
… split

Move all state and private methods into a non-copyable impl class
(detail::reservation_aware_resource_adaptor_impl). The public wrapper
now inherits from cuda::mr::shared_resource<impl>, making it copyable
and movable via reference counting. This satisfies the CCCL resource
concept requirements for any_resource, enabling use with device_buffer
and device_uvector in RMM 26.06.

Follows the same pattern used by RMM's own pool_memory_resource and
tracking_resource_adaptor.
…ctations

Remove extra closing brace that broke SCENARIO block structure. Fix the
'exceeds reservation' test case which previously expected overflow with
two chunk_size allocations that exactly filled the reservation — add a
third allocation to actually trigger overflow, and correct the expected
total_allocated_bytes to reservation_size + chunk_size.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants