ssd-offloading 开源项目推荐（9 个）

FonaTech /Project_Chronos

⚡ 通过前瞻预测和异步DMA预取进行零停滞MoE推断。针对 SSD 输入输出优化，采用混合 MLA+滑动窗口关注。

arnavprabhu /MacSeek

From-scratch Metal inference engine that runs DeepSeek-V4.1-Flash (233 GB GGUF) with 200K context on a 64 GB Mac — 7x faster prefill, 12x faster decode than llama.cpp. OpenAI-compatible server + web UI.

jiro-prog /llamacpp-moe-expert-cache

125B MoE (Qwen3.8-Flash-Next) on 16GB RAM + 8GB GPU: expert cache, next-layer read-ahead and GPU keep-alive for llama.cpp on Windows

nervitesi /NInfer-M1-Flash

A llama.cpp fork that runs Qwen3.8-Flash-Next (180B MoE, 97.5 GB at true Q4) on a 64 GB M1 Max at ~30 tok/s: experts streamed from the SSD, Metal 4, sparse attention, MTP drafts. 1.9-6x faster than other engines on the same file.

sjdthree /kimi-k3-mlx-macmini

Run the full 2.8T-parameter Kimi-K3 checkpoint from SSD on a 64 GB M4 Pro Mac mini: all 93 layers, per-expert streaming, native MLX/Metal. Slow (~32 s/token) but complete.

arnavprabhu /deepseek-v4-flash-mac

Run DeepSeek-V4.1-Flash (551B MoE, IQ2_XXS) locally on Apple Silicon with 128K context, even though it's bigger than RAM. Patched llama.cpp Metal runtime streams experts from SSD with a GPU expert cache. Tested on M5 Pro 64 GB: 252 tok/s prefill, 16.3 tok/s decode.

jiro-prog /kimi-k3-ssd-streaming

Kimi K3 (2.8T MoE) on a 16GB-RAM Windows PC: SSD streaming for llama.cpp, 45 -> 4.5 s/token

aarushgoradia /llama.cpp-moe-streaming

senior thesis: trillion-param moe inference on a few ordinary pcs, with experts streamed nvme → dram → gpu. fork of llama.cpp.

n0rpan /snapdragon-engine

Purpose-built Rust inference engines for Snapdragon X CPUs: a static-plan MoE engine with MTP verify lanes, SSD expert streaming for a 125B MoE in 32 GB, and an int8 image engine.