sliding-window-attention 开源项目推荐（12 个）

FonaTech /Project_Chronos

⚡ 通过前瞻预测和异步DMA预取进行零停滞MoE推断。针对 SSD 输入输出优化，采用混合 MLA+滑动窗口关注。

hkproj /mistral-llm-notes

Notes on the Mistral AI model

XunhaoLai /ring-sliding-window-attention

Ring sliding window attention implementation with flash attention

RubensZimbres /TPU_Inference

Measurement harness for the sliding window attention premium in the vLLM TPU Ragged Paged Attention v3 kernel: per layer decode cost, block size control, throughput, and goodput for Gemma 4 31B on TPU v6e.

rickey1990 /qk-relational-architecture

Experimental headless QK relational architecture exploring both full-pair and linear fixed-window distance-aware reading, with a separate strict reusable multi-hop relational mechanism.

Ashutosh0x /claude-rust

A high-performance terminal-integrated LLM engine in Rust

Andgihat /strata-flint

High-performance native C++/CUDA inference engine for Flint 40B Preview MoE (256 experts). Batched Tensor Core prefill (1,238 tok/s), CUDA Graph decode (90 tok/s), 3-tier sliding KV cache (256K context on 16GB GPUs), adaptive expert swapping.

atandra2000 /GPT-OSS-Lite

Faithful from-scratch PyTorch reproduction of OpenAI's GPT-OSS architecture (sliding/full attention alternation, learned attention sinks, YaRN 128K, top-2-of-8 MoE), scaled to Chinchilla-optimal 502M total / 247M active training on a single A100 80GB

Pavelpuff /gpt-oss-20b-core

PyTorch implementation of an efficient decoder-only LLM featuring RoPE, GQA, Sliding Window Attention, and MoE with SwiGLU routing.

sachncs /beacon

Training-free Sliding Window Attention with sinks: a reproduction of 'Sliding-window beats linear attention' (Jolicoeur-Martineau et al., 2026) for decoder-only LMs.

JohnScheuer /sliding-window-attention-kv-bench

Analytical benchmark for sliding window attention KV cache management: quality vs window size tradeoffs, SWA vs eviction comparison, prefix sharing interaction, and operational window recommendations across four attention distributions

alhussein-jamil /ViTHilbertPatches

Vision Transformer that orders patches on a Hilbert curve so local attention keeps their 2D neighborhood.