linear-attention 开源项目推荐（139 个）

BlinkDL /RWKV-LM

RWKV（发音为 RwaKuv）是一种具有出色 LLM 性能的 RNN，也可以像 GPT transformer 一样直接训练（可并行化）。我们目前处于 RWKV-7 “Goose”。因此它结合了 RNN 和 transformer 的优势——出色性能、线性时间、恒定空间（无需 kv-cache）、快速训练、无限 ctx_len，以及免费的句子嵌入。

FareedKhan-dev /kimi-k3-in-c

2.78万亿参数Kimi K3在8.24 GB RAM中的单个中央处理器上运行推理。便携式C99：无BLAS、无框架、无图形处理器。

thu-ml /SLA

SLA：通过精细可调稀疏线性注意力超越扩散变换器中的稀疏性

weigao266 /Awesome-Efficient-Arch

Speed Always Wins: A Survey on Efficient Architectures for Large Language Models

Interactive-Active-AI /NextTransformer

🌅 A comprehensive, cutting-edge open-source framework for next-generation Transformers/LLM, with compute-matched benchmarking and a platform for autonomous architecture research.

happinesslz /LION

[NeurIPS 2024] Official code of ”LION: Linear Group RNN for 3D Object Detection in Point Clouds“

zhouyuan888888 /CARE-Transformer

CARE Transformer: Mobile-Friendly Linear Visual Transformer via Decoupled Dual Interaction

DAGroup-PKU /MHLA

[ICLR 2026🔥] MHLA: Restoring Expressivity of Linear Attention via Token-Level Multi-Head

Intellindust-AI-Lab /GTR

GTR: Gated Token Recurrence for Efficient Dense Prediction

lucidrains /taylor-series-linear-attention

Explorations into the recently proposed Taylor Series Linear Attention

lucidrains /agent-attention-pytorch

Implementation of Agent Attention in Pytorch

lironui /Multi-Attention-Network

遥感图像的语义分割

keshik6 /grafting

[NeurIPS 2025 Oral] Official Code for Exploring Diffusion Transformer Designs via Grafting

NX-AI /KronLSTM

Flash-linear-attention compatible KronLSTM model and kernels by NXAI.

lironui /MAResU-Net

遥感图像的语义分割

HaoyiZhu /Piecewise-Taylor-Attention

Piecewise-Taylor Attention

lucidrains /autoregressive-linear-attention-cuda

CUDA implementation of autoregressive linear attention, with all the latest research findings

TimRots /kimi3

Independent from-scratch implementation of the Kimi K3 architecture (arXiv:2607.24653v1): KDA, NoPE, latent-space MoE, and the systems co-designs. Table 1 reproduced to 0.09%.

tommasocerruti /linear-attention-architectures

Linear Attention Architectures: Mechanisms, Trade-offs, and Cross-Layer Routing (Technical Report)

lucidrains /bdh-cq

Implementation of BDH-CQ: In-Context Learning with Recurrent Latent Reasoning, proposed by Pathway Research