Skip to content

FP16 KV cache with packed half2 split-KV attention#138

Open
orionpapadakis wants to merge 5 commits into
mainfrom
feat/fp16-kv-cache
Open

FP16 KV cache with packed half2 split-KV attention#138
orionpapadakis wants to merge 5 commits into
mainfrom
feat/fp16-kv-cache