Xnhyacinth/Awesome-LLM-Long-Context-Modeling：必读基于LLM的长上下文建模的论文和博客🔥

项目介绍

Xnhyacinth/Awesome-LLM-Long-Context-Modeling 是一个托管在 GitHub 上的开源项目，采用 MIT License 许可证，目前获得 2.2k 个 Star、105 次 Fork。

仓库全名: Xnhyacinth/Awesome-LLM-Long-Context-Modeling

README 摘要

Large Language Model Based Long Context Modeling Papers and Blogs 📝 Survey Paper | 📄 Paper List | 🏠 Homepage | 📚 Notes | ⭐ GitHub This repository curates papers and blogs on long-context language modeling, covering surveys; efficient attention; KV-cache optimization; recurrent transformers and state-space models; position encoding & length extrapolation; long-context training and mid-training; long-term memory…

完整内容见 GitHub 原始 README 或本页。

常见问题

Awesome-LLM-Long-Context-Modeling 可以免费商用吗？

如何获取 Awesome-LLM-Long-Context-Modeling 的源码？

可以直接克隆仓库：git clone https://github.com/Xnhyacinth/Awesome-LLM-Long-Context-Modeling.git，默认分支为 main；安装与运行步骤见仓库 README。

Awesome-LLM-Long-Context-Modeling 最近还在维护吗？

仓库最近一次更新为 2 周前，当前没有未关闭的 Issue；Star 数为 2.2k，可作为社区活跃度的参考。

Large Language Model Based Long Context Modeling Papers and Blogs

📝 Survey Paper | 📄 Paper List | 🏠 Homepage | 📚 Notes | ⭐ GitHub

This repository curates papers and blogs on long-context language modeling, covering surveys; efficient attention; KV-cache optimization; recurrent transformers and state-space models; position encoding & length extrapolation; long-context training and mid-training; long-term memory; retrieval-augmented generation; in-context learning; context and model compression; long reasoning (long CoT); long video & image; long-horizon agents; long-text generation; inference acceleration; benchmarks & evaluation; and technical reports.

🔥 Must-read papers for LLM-based Long Context Modeling.

🔥⚡🔥 Thanks for all the great contributors on GitHub!

🚀🤝🚀 I have the privilege of joining [LCLM-Horizon] and collaborating with them on providing a very complete and comprehensive scholarly survey (A Comprehensive Survey on Long Context Language Modeling) and repository (A-Comprehensive-Survey-For-Long-Context-Language-Modeling) dedicated to Long Context Language Modeling. I look forward to collaborating with them to advance research and deepen understanding in this area!

Taxonomy at a glance

flowchart LR LCLM["Long-Context Modeling"] LCLM --> A["Attention & KV Cache"] LCLM --> T["Training & Alignment"] LCLM --> M["Memory & RAG"] LCLM --> C["Compression"] LCLM --> R["Reasoning & Generation"] LCLM --> V["Multimodal / Video"] LCLM --> E["Evaluation & Acceleration"] A --> A1["Sparse / Linear / IO-aware Attention"] A --> A2["Eviction / Quantization / Offloading"] T --> T1["Continual Pretraining / Long-SFT"] T --> T2["Adaptation & RL for Long Context"] T --> T3["Mid-Training: Data, Recipes & Scaling"] M --> M1["Long-Term Memory"] M --> M2["RAG / Hybrid Long-Context"] C --> C1["Context Compression"] C --> C2["Model Compression"] R --> R1["Long CoT"] R --> R2["Long-Form Text Generation"]

If you find our repository and survey useful for your research, please consider citing the following paper:

@article{liu2025comprehensive,
  title={A Comprehensive Survey on Long Context Language Modeling},
  author={Liu, Jiaheng and Zhu, Dawei and Bai, Zhiqi and He, Yancheng and Liao, Huanxuan and Que, Haoran and Wang, Zekun and Zhang, Chenchen and Zhang, Ge and Zhang, Jiebin and others},
  journal={arXiv preprint arXiv:2503.17407},
  year={2025}
}

📢 News

Week Papers

[2026.10.23]
[2026.10.22]
[2026.10.21]
[2026.10.20]
- Paper: Scan or Index? Large Evidence Coverage, Answer Quality, and Cost in Long-Document RAG
- Paper: Cache Engineering for Agents
[2026.10.19]
[2026.10.18]
- Paper: DiaLong: Benchmarking Reminiscence in Long-Context Dialogues
- Paper: The Bottleneck Is Retrieval, Not Reading: Diagnosing Evidence-Based Generation with RAG for Long Context Question Answering
[2026.10.17]
- Paper: TideKV: Delayed KV Eviction via Layer-Wise Chunked Prefill for Memory-Efficient Long-Context LLM Inference
[2026.10.16]
- Paper: Towards Looped Models Done Right, Part I: Topology, Input Injection, Recurrent-State Design
- Paper: DLite: History-free Speculative Decoding for Agentic AI
[2026.10.15]
[2026.10.14]
[2026.10.13]

Month Papers

[2026.10.12]
- Paper: FreeSolo: Linear-time Sparse Attention with Zero-Shot Length Extrapolation
- Paper: Elephant in the Fridge: Constant-Memory Frame Packing for Long Video Understanding
[2026.10.11]
- Paper: History Replays Itself: Unleashing Long-Context Capability in Linear Attention
- Paper: Apertus 1.5: From Open Text Models to Open Multimodal Reasoners
[2026.10.10]
- Paper: SPARE: Shared Perception and Adaptive Retrieval of Evidence for Streaming Video Understanding
- Paper: SAFE-KV: Causal Suffix Repair for Behavior-Preserving Non-Prefix KV Cache Reuse in RAG
- Paper: KV4-RL: Unlocking Faster Asynchronous Rollouts for RLVR with INT4 KV Cache Quantization
[2026.10.09]
- Paper: Constructing Pareto Frontiers of Optimal Reasoning Termination Methods for Overthinking in Reasoning Models
- Paper: REMORY: Learning Residual Memory for Context Compaction
[2026.10.08]
- Paper: FastBench: Can Streaming VLMs Perceive High-Dynamic Real-World Streams?
- Paper: From Retrieval to Reconstruction: Constructing Evolvable Cognitive Memory for Long-Term Dialogue
- Paper: Thinking Inertia: LLMs Keep Thinking When Told Not To
- Paper: V-CoLA: Vision Token Compression with Linear Attention
[2026.10.07]
- Paper: VideoEvolve: Co-Evolving Memory and Retrieval for Long Video Understanding
- Paper: Mechanics of Long-Context Hybrid Models Part 1.1: From Hybrid Attention to Hybrid Position
[2026.10.06]
- Paper: UNREAL: Unifying Retrieval and Long-Context with a Single Model
[2026.10.05]
- Paper: ReMem: Streaming Video Understanding With Long Context Retention
- Paper: Behavior-Preserving KV Cache Compression
[2026.10.03]
- Paper: WavePrune: One period is often enough for RoPE
- Paper: Rethinking Long-Video Efficiency: A Joint Allocation Perspective on Frames, Pixels, and Front-End Latency
[2026.10.02]
- Paper: LongSocialBench: Do Long-Context LLMs Understand Online Discussion Threads?
[2026.10.01]
- Paper: OneStreamer: Unifying Perception, Memory, and Proactive Response in Streaming Video Interaction
- Paper: How Much Can Language Models Gain from Test-Time Computation?
[2026.09.30]
- Paper: ReLiveGym: Evaluating Long-Lived Agents over Weeks of Replayed Reality
- Paper: RoPE at the End of Its Rope? Theory, Diagnosis, and Mitigation of Long-Context Failures
- Paper: Persistent Context Graphs for Efficient Memory Compaction in LLM Agents
- Paper: SparseEngine: Sparse-First Inference Engine
- Paper: Switching Linear Attention
[2026.09.29]
- Paper: LongHarness Bench: Stress-Testing Language Model Harnesses for Long-Context Reasoning
- Paper: Triadic Linear Attention: Three-Dimensional Recurrent States for Long-Context Sequence Modeling
- Paper: PolyOCR-Venus: Unified OCR Foundation Models for Text-Centric Visual Intelligence
- Paper: Beyond the Timeline: Augmenting Long-Video Memory with Grounded Entity Biographies
- Paper: WUSH-KV: KV Cache Quantization with Data-Adaptive Transforms
- Paper: APM-Bench: Benchmarking Cross-session Persistent Memory for Egocentric Streaming Video Assistants
- Paper: VideoLoop: Looped Working Memory Against Semantic Thrashing in Long-Form Video Agents
[2026.09.28]
- Paper: Scaling Long-Form Story Generation via Narrative State Tracking
- Paper: RoPE is Dead, Long Live RoPE: Towards Scalable Data-aware Positional Encodings
- Paper: Improving Test-Time Scaling with Adaptive Looped Transformers
- Paper: Fractional State Space Transition for Long Sequence Modeling
- Paper: Beyond Selection: Token Parameterization for Extreme Visual Token Compression
- Paper: Periodic Weak Spots: Phase Sensitivity from Chunked KV-Cache Compression
- Paper: Xiaomi-OCR-0 Technical Report
[2026.09.26]
- Paper: Shared Worlds, Private Minds: Structured Memory for Long-Form Writing as World Creation
- Paper: Prefill-Free Cross-Family KV Cache Transfer for Heterogeneous Multi-Agent LLMs
- Paper: MassAlloc Attention: Let Attention Allocate Its Own Compute
- Paper: DoSA: Dots Sparse Attention
[2026.09.25]
- Paper: Highlight-Then-Summarize: Learning to Compress Evidence for Long-Context Understanding
- Paper: Block Sparse Attention with Log-Linear Complexity
[2026.09.24]
- Paper: MILO: Efficient Many-shot In-Context Learning with Block-wise Low-rank Compression
[2026.09.23]
- Paper: KITE: KV-Invariant Transformer Expansion for Efficient Agentic LLM Scaling
[2026.09.22]
- Paper: Compressing Long Context into Answer-Aligned Memory Embeddings for LLM Inference
- Paper: CompKV: Compensation-Aware KV Selection for Long-Context LLM Inference
- Paper: Knowledge Pull Requests for Continual Document Authoring
- Paper: Qwen3.8-Omni: Towards Native Omni-Modal Agents
- Paper: Disaggregated Quantization: Specializing LLM Prefill and Decode
[2026.09.21]
- Paper: Jev-Mem: System-One-Controlled Agentic Memory for Efficient AI Agents
- Paper: DolphinBench: Mapping the Pareto Frontier of Agent Memory
[2026.09.20]
- Paper: RPMem: Learning Long-Term Recurrent Parametric Memory Across Sessions for LLM Agents
- Paper: Global Ranks Survive, Selected Heads Shift: BOS-Sink Topology under 4-bit Weight-Only Quantization
[2026.09.17]
- Paper: DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression
- Paper: When2Think: Learning When and How Much to Reason
- Paper: When EOS Tokens Disagree: Understanding Length Inflation in On-Policy Distillation
[2026.09.16]
- Paper: ASPIRE: Asynchronous Batched Self-Speculative Decoding for Long-Context LLM Inference
[2026.09.13]
- Paper: SpectralShift: Effective Context Window Extension of Gated DeltaNet via Spectral Reparameterization
[2026.09.11]
- Paper: SAS: Simple Attention Sparsification via End-to-End Optimization of Context Ranking
[2026.09.10]
- Paper: FlexComp: One Model for Every Ratio in Context Compression
[2026.09.07]
- Paper: On the Recall Scaling Laws in Mamba: A Theoretical and Mechanistic Study via Hashing
- Paper: Online Draft Co-Training for Speculative Decoding in Large-Scale, Long-Context RL Post-Training
- Paper: ReactVAU: A Slow-Fast Decoupled Framework for Streaming Video Anomaly Understanding
[2026.09.06]
- Paper: PARSER: Read in Parallel, Reason in Depth for Long-Context LLM Agents
[2026.09.04]
- Paper: BeaconKV: Key-Value Cache Compression Guided by Beacon Queries for Efficient Large Reasoning Model Inference
[2026.09.03]
- Paper: Why Gated DeltaNet Survives 4-Bit Quantization: NVFP4 W4A4 for the Recurrent Half of a Hybrid 27B LLM
- Paper: Beyond Retrieval: Progressive Latent Memory Evolution for Streaming Video Understanding
- Paper: Random Attention: Rethinking KV Cache Eviction for Efficient Reasoning
[2026.09.02]
- Paper: Language Models Can Control Their Own Attention
[2026.09.01]
- Paper: Instella-MoE Technical Report
- Paper: Compile, Don’t Memorize: A Context Compilation Architecture (CCA) for In-Context Learning
- Paper: LatentPress: Context Compression Beyond Text and Vision
[2026.08.28]
- Paper: Sliding-window beats linear attention
- Paper: ContextPilot: Teaching Agents for Proactive Context Management via Fine-grained RL

📜 Papers

Paper entries live under papers/ so this README stays under GitHub’s homepage size limit. For an interactive chapter reader (search + in-page paper cards), open the project homepage.

Attention, recurrence & systems

Training, position & memory

Compression, reasoning & multimodal

Evaluation & reports

Acknowledgements

Please contact me if I miss your names in the list, I will add you back ASAP!

Awesome-LLM-Long-Context-Modeling

项目介绍

README 摘要

常见问题

关于

简要分析

贡献者

Large Language Model Based Long Context Modeling Papers and Blogs

Contents

📢 News

Week Papers

Month Papers

📜 Papers

Acknowledgements

Contributors

Star History

评论

Awesome-LLM-Long-Context-Modeling

项目介绍

README 摘要

常见问题

相关仓库

关于

简要分析

贡献者

Large Language Model Based Long Context Modeling Papers and Blogs

Contents

📢 News

Week Papers

Month Papers

📜 Papers

Acknowledgements

Contributors

Star History

评论