fix(qwen3next): keep big-page state cache pinned across shm serialization - #1422
Merged
hiworldwzj merged 1 commit intoAug 4, 2026
Merged
Conversation
Contributor
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
…tion write_to_shm serializes this object with ForkingPickler so other processes can share it. During that dump, torch migrates each CPU tensor's storage in-place into shared memory, which degrades the pinned (cudaHostAlloc) big-page state cache to unpinned shm in the local process; the linear attention small-page copy Triton kernel then rejects the pointer with: ValueError: Pointer argument cannot be accessed from Triton (cpu tensor?) Temporarily drop linear_att_big_page_buffers around the super() call so the dumper never sees it. The local pinned allocation is preserved, and cross-process consumers (pd trans / dp prompt cache fetch), which do not use the CPU-side big-page state cache, are unaffected.
sufubao
force-pushed
the
fix/linear-att-pinned-shm-getstate
branch
from
August 4, 2026 08:53
c6d1fb1 to
148c9a2
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
问题
PD / 多进程(
write_to_shm共享 mem_manager)场景下,linear attention 模型(qwen3next)开启 cpu cache 后,碎页 offload 时崩溃:根因
linear_att_big_page_buffers在LayerCache里以pin_memory=True(cudaHostAlloc)分配,此时 Triton kernel 可以直接访问该 pinned host 内存(已验证 pinned tensor 可被 Triton 读写)。但
write_to_shm通过ForkingPickler.dumps(self)序列化Qwen3NextMemManager时,torch 会把其 CPU tensor 的 storage 原地迁移到共享内存(shm mmap)。这一迁移把本进程持有的大页 state cache 从 pinned 退化成了普通 unpinned 的 shm。之后碎页 offload 调用copy_linear_att_state_to_linear_att_state(一个 Triton kernel)携带该指针启动,Triton 拒绝非 pinned 的 CPU 指针,抛出上述错误。关键点:失败的不是「CPU 指针喂给 GPU kernel」,而是 pinned 属性在 shm 序列化过程中被丢失。单进程(不走
write_to_shm)不会触发。修复
重写
__getstate__,序列化时把linear_att_big_page_buffers置为None,避免 torch 迁移它的 storage,从而保住本进程的 pinned 属性。为什么安全
跨进程消费方(pd trans 进程 / dp prompt cache fetch)并不使用 cpu 侧的
linear_att_big_page_buffers,序列化时剔除不会影响它们;而本进程在反序列化后仍持有原始 pinned buffer(__init__时分配,未经过 pickle 重建),pinned 属性得以保留。验证
--enable_cpu_cache+ linear attention 模型跑碎页流量,prefill 节点不再崩溃;upstream/main 同配置必现崩溃。