Pinned Loading

1. xlite-dev/lite.ai.toolkit xlite-dev/lite.ai.toolkit Public

   🛠 A lite C++ AI toolkit: 100+🎉 models (Stable-Diffusion, FaceFusion, YOLO series, Det, Seg, Matting) with MNN, ORT and TensorRT.

   C++ 4.1k 739

2. vllm-project/vllm vllm-project/vllm Public

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Python 47k 7.3k

3. xlite-dev/Awesome-LLM-Inference xlite-dev/Awesome-LLM-Inference Public

   📚A curated list of Awesome LLM/VLM Inference Papers with codes: WINT8/4, FlashAttention, PagedAttention, MLA, Parallelism etc.

   Python 4k 276

4. PaddlePaddle/FastDeploy PaddlePaddle/FastDeploy Public

   ⚡️An Easy-to-use and Fast Deep Learning Model Deployment Toolkit for ☁️Cloud 📱Mobile and 📹Edge. Including Image, Video, Text and Audio 20+ main stream scenarios and 150+ SOTA models with end-to-end…

   C++ 3.2k 477

5. xlite-dev/LeetCUDA xlite-dev/LeetCUDA Public

   📚LeetCUDA: Modern CUDA Learn Notes with PyTorch for Beginners🐑, 200+ CUDA/Tensor Cores Kernels, HGEMM, FA-2 MMA etc.🔥

   Cuda 4.1k 440

6. xlite-dev/ffpa-attn xlite-dev/ffpa-attn Public

   📚FFPA(Split-D): Extend FlashAttention with Split-D for large headdim, O(1) GPU SRAM complexity, 1.8x~3x↑🎉 faster than SDPA EA.

   Cuda 171 7