mxfp4 开源项目推荐（45 个）

FareedKhan-dev /kimi-k3-in-c

2.78万亿参数Kimi K3在8.24 GB RAM中的单个中央处理器上运行推理。便携式C99：无BLAS、无框架、无图形处理器。

intel /auto-round

一种面向高精度低比特 LLM 推理的 SOTA 量化算法，无缝优化 CPU/XPU/CUDA，支持多种数据类型，并完全兼容 vLLM、SGLang 和 Transformers。

tsaipifong /whirl-llm

WHIRL (Windows HIP Inference for RDNA LLMs): native Windows C++/HIP LLM inference engine for the AMD Ryzen AI Max+ 395 (Radeon 8060S) and Radeon AI PRO R9700. Unzip and run, only the AMD driver.

melonmath /microkimi

The Kimi K3 and DeepSeek-V4-Flash-0731 architectures, reimplemented from scratch in pure Rust (zero dependencies) and verified 1:1 against the official code.

yeungtuzi /Xiaotu-LLM

XTU = X Transformers Unity — hybrid (CPU-expert + GPU) inference acceleration for vLLM: mainline plugin + CPU MoE engine (AVX512-VNNI/BF16 MXFP4/FP8), long-prefill GPU streaming, SM80 (A100) port.

Capicua25x /vllm-rocm-rdna4

vLLM for AMD RDNA4 (gfx1201): Radeon AI PRO R9700 & RX 9070 XT — native MXFP4 linear kernel + spec-decode verify fix, carried as rebased branches (fork-carry model)

drmihirbrahme /midge

Tiny engine, immense models — run large MoE LLMs (gpt-oss, Mixtral, Qwen3-MoE) on ordinary machines by streaming experts from disk. OpenAI-compatible server with tool calling + hybrid cloud relay; CPU, Apple Silicon (MLX) & CUDA.

xltzsoft /deepseek-v4-sm89

面向 8×RTX 4090（SM89）的 DeepSeek-V4-Flash-0731 MXFP4 SGLang 实验分支

Maknee /Kimi-K3.cpp

Run Kimi-K3 (2.8T MoE) from SSD with a tiny readable C++17 engine—native MXFP4, io_uring, and O_DIRECT.

outerreaches /moonshine

Run the official Kimi K3 MoE checkpoint on one 128 GB AMD Strix Halo box. ROCm-resident static weights, MXFP4 experts streamed from NVMe via io_uring. C engine, chat client, OpenAI-compatible server. Very experimental.

SSHdotCodes /deepseek-v4.1-flash-pro6000

DeepSeek-V4.1-Flash-1017 at native precision on one RTX PRO 6000 + 128 GB RAM: 31.6 tok/s decode, 1M context. ds41d engine with VRAM/RAM/NVMe expert tiers, GPUDirect Storage and DSpark speculative decoding

bricesommers /apus-deepseek-v4-flash

Local inference engine for DeepSeek-V4-Flash (284B MoE, MXFP4 experts streamed from disk) on consumer hardware — C11, zero deps, bit-exactness gated. macOS / Linux / Windows.

pjordanandrsn /experts4bit-qlora

Fine-tune and serve large Mixture-of-Experts models on one GPU, with 4-bit experts in GPU memory, RAM or SSD.

fei121 /Qwen3-8B-Quantization

Qwen3-8B quantization study across vLLM, TensorRT-LLM, AutoRound, INT8, and MXFP4

pjordanandrsn /grouped-nf4-gemm

4-bit GPU kernels that compute directly on packed Mixture-of-Experts weights, for training and inference.

heyobi /LazyLora

Out-of-core LoRA fine-tuning and expert-routing measurement for Kimi K3 (2.78T MoE) on a 7.6 GB laptop: 93 layers streamed from a USB disk, forward checked against an independent C implementation, gradients checked by finite differences

randzero /quantkit

A dependency-light, transformers-free toolkit for data-free weight-only quantization (FP8 blockwise, MXFP8, MXFP4), with file-level multi-node support and HF / compressed-tensors output.

dardeaw /strata-mxfp4-rdna

MXFP4 for RDNA4 (gfx12): W4A8 FP8 expert path, LDS FP8 GEMM, grow config (dense on GPU, VRAM to the expert cache)

KaiDong-coder /hotset

gpt-oss-120B inference on one 48 GB GPU: model-exact placement, MXFP4 mul_mat_id fast paths, and a routing-traced byte model for what comes next.

sjdthree /kimi-k3-mlx-macmini

Run the full 2.8T-parameter Kimi-K3 checkpoint from SSD on a 64 GB M4 Pro Mac mini: all 93 layers, per-expert streaming, native MLX/Metal. Slow (~32 s/token) but complete.