Conversation
Ceng23333
requested review from
a team,
Ziminli,
kilinchange,
voltjia and
wooway777
qinyiqun
reviewed
Apr 14, 2026
qinyiqun
reviewed
Apr 14, 2026
qinyiqun
reviewed
Apr 15, 2026
|
|
||
| void run(void *planned_meta) { | ||
| #ifdef ENABLE_FLASH_ATTN | ||
| #ifdef ENABLE_NVIDIA_API |
Collaborator
There was a problem hiding this comment.
刚才轶群说，QY也需要这个，所以是不是合并一下把判断去掉就行了？还是说把三个都加上才能不影响其他平台？
qinyiqun
reviewed
Apr 15, 2026
| #include <stdexcept> | ||
|
|
||
| #ifdef ENABLE_FLASH_ATTN | ||
| #if defined(ENABLE_NVIDIA_API) || defined(ENABLE_METAX_API) |
Collaborator
There was a problem hiding this comment.
没加的话，就是默认需要，因为qy默认走跟nv一样的路径
wooway777
reviewed
Apr 16, 2026
wooway777
requested changes
Apr 21, 2026
|
|
||
| void run(void *planned_meta) { | ||
| #ifdef ENABLE_FLASH_ATTN | ||
| #ifdef ENABLE_NVIDIA_API |
|
|
||
| void run(void *planned_meta) { | ||
| #ifdef ENABLE_FLASH_ATTN | ||
| #ifdef ENABLE_NVIDIA_API |
Collaborator
There was a problem hiding this comment.
刚才轶群说，QY也需要这个，所以是不是合并一下把判断去掉就行了？还是说把三个都加上才能不影响其他平台？
| auto out_tensor = infinicore::adaptor::to_aten_tensor(p->out); | ||
| // Paged KV caches must be contiguous for flash-attn; avoid extra copies for q/metadata when already dense. | ||
| auto out_at = infinicore::adaptor::to_aten_tensor(p->out); | ||
| const bool out_need_copy_back = !out_at.is_contiguous(); |
Collaborator
There was a problem hiding this comment.
这两个contiguous nvidia本身是不是不需要做？
另外应该改成先用我们的contiguous再转成aten tesnor
| auto v_cache = infinicore::adaptor::to_aten_tensor(p->v_cache); | ||
| #elif defined(ENABLE_QY_API) | ||
| #elif defined(ENABLE_QY_API) || defined(ENABLE_METAX_API) | ||
| auto k_cache = infinicore::adaptor::to_aten_tensor(p->k_cache).contiguous(); |
Collaborator
There was a problem hiding this comment.
先用我们的contiguous再转成aten tesnor
| VarlenFlashPrepared t; | ||
| // Varlen flash-attn: keep k/v contiguous for dense/paged layout; avoid extra copies for q/metadata when already dense. | ||
| t.q = infinicore::adaptor::to_aten_tensor(p->q); | ||
| t.k = infinicore::adaptor::to_aten_tensor(p->k).contiguous(); |
Collaborator
There was a problem hiding this comment.
先用我们的contiguous再转成aten tesnor
qinyiqun
approved these changes
Apr 22, 2026
Signed-off-by: Ceng23333 <441651826@qq.com>
Signed-off-by: Ceng23333 <441651826@qq.com>
Signed-off-by: Ceng23333 <441651826@qq.com>
wooway777
approved these changes
Apr 27, 2026
Signed-off-by: Ceng23333 <441651826@qq.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
#1117