kv-cache 开源项目推荐（1,378 个）

LMCache /LMCache

LMCache：以最快的 KV 缓存层为你的 LLM 强力加速

bojieli /ai-infra-book

《深入理解 AI Infra：量化分析与系统设计》（李博杰 著）开源书稿：从硬件约束和模型架构出发，量化推导 LLM 推理与训练系统设计。含全书正文、PDF、配套计算工具与实验

HDT3213 /godis

A Golang implemented Redis Server and Cluster. Go 语言实现的 Redis 服务器和分布式集群

syv-ai /HyperQwen

Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.

AI-Efficiency /Awesome-Model-Quantization

A curated collection of papers, benchmarks, surveys, and tools for model quantization, covering low-bit networks, LLMs, multimodal and generative models, vector and lattice quantization, and efficient deployment.

datawhalechina /zero-to-sglang

官方 SGLang x Datawhale LLM 推理课程：理解推理，从零构建 mini-sglang，然后阅读真实 SGLang 源码并完成你的第一个 PR。提供英文和中文版本。

Zefan-Cai /KVCache-Factory

Unified KV Cache Compression Methods for Auto-Regressive Models

NVIDIA /kvpress

LLM KV cache compression made easy

Anbeeld /beellama.cpp

KVarN, KV cache precision tail, low-bit quants in llama.cpp for longer context of better precision in the same VRAM

harleyszhang /llm_note

LLM笔记，包括模型推理、变换器模型结构和LLM框架代码分析笔记。

thu-nics /C2C

[ICLR'26] The official code implementation for "Cache-to-Cache: Direct Semantic Communication Between Large Language Models"

pegainfer-project /pegainfer

Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

raymin0223 /mixture_of_recursions

Mixture-of-Recursions: Learning Dynamic Recursive Depths for Adaptive Token-Level Computation (NeurIPS 2025)

therealoliver /Deepdive-llama3-from-scratch

分步实现llama 3推理，掌握核心概念，掌握流程推导，实现代码。

zengxiao-he /tessera

从教师到瓦片-从头开始的LLM蒸馏和服务引擎：自定义Triton/CUDA内核、FSDP蒸馏、页面KV连续收件箱、推测性解码、Rust网关、JAX Oracle和可解释性工具。

FMInference /H2O

[NeurIPS'23] H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models.

huawei-csl /KVarN

KVarN is a native vLLM KV-cache quantization backend for your agents: 3-5x more context, throughput above FP16, and FP16-level accuracy. Calibration-free, one flag.

Zefan-Cai /Awesome-LLM-KV-Cache

Awesome-LLM-KV-Cache: A curated list of 📙Awesome LLM KV Cache Papers with Codes.

jjiantong /Awesome-KV-Cache-Optimization

[ACL 2026] Towards Efficient Large Language Model Serving: A Survey on System-Aware KV Cache Optimization

quantumaikr /quant.cpp

LLM推理，具有7倍更长的上下文。纯C，零依赖。无损KV缓存压缩+单头库。