Fast CAGRA Index Merge - #2352
Conversation
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
|
/ok to test 2c54392 |
aamijar
added
non-breaking
|
/ok to test |
|
/ok to test |
|
/ok to test |
|
/ok to test |
Co-authored-by: Artem M. Chirkin <9253178+achirkin@users.noreply.github.com>
Co-authored-by: Artem M. Chirkin <9253178+achirkin@users.noreply.github.com>
1b8558a to
653e8a0
Compare
The graph merge PR (#2352) uses some of the kernels defined by CAGRA, and binary size takes a hit from them not appearing in a TU where they can be shared by both CAGRA and Fastener. Authors: - Ben Landrum (https://github.com/landrumb) Approvers: - Divye Gala (https://github.com/divyegala) URL: #2412
dantegd
left a comment
dantegd
left a comment
There was a problem hiding this comment.
Have only the small concerns, besides it things look solid!
Also, it looks like a few working artifacts may have slipped into the PR that need to be removed before merge: PR_2352_REVIEW_WORK.md, pipnn.pdf and FASTENER_PR_PLAN.md, right?
achirkin
left a comment
achirkin
left a comment
There was a problem hiding this comment.
Thanks for the refactoring! One problem with the switch to cublasLtMatmul (see below).
| decltype(&cublasLtMatrixLayoutDestroy)>; | ||
|
|
||
| cublasLtMatmulDesc_t operation_raw = nullptr; | ||
| RAFT_CUBLAS_TRY(cublasLtMatmulDescCreate(&operation_raw, GEMM_COMPUTE_TYPE, CUDA_R_32F)); |
There was a problem hiding this comment.
Please destroy the descriptors and matrix layouts after you use them, (otherwise, this is a memory leak currently).
Have a look at how it's one in raft: https://github.com/NVIDIA/raft/blob/main/cpp/include/raft/linalg/detail/cublaslt_wrappers.hpp
I think, the best solution though would be to create a tiny PR in raft to add the corresponding wrapper/helper for the configuration you need and then use it here.
There was a problem hiding this comment.
Added raft PR for this: NVIDIA/raft#3106
5 participants
This PR implements the Fastener graph merge operation. This PR supports merging for
float,half,int8, anduint8dtypes, and euclidean distances.There was originally specialization to use int8 GEMM for the integer types, but I ran into portability issues on Ada and it turns out that using the same f32 path for both is simpler and not substantially slower. Currently investigating switching the unified path to TF32 to use tensor cores.
The core logic resides in
cagra_merge_scaffold.cuh.This PR adds 7.64 MiB to
libcuvs.so, a 2.94% increase.