fp4 开源项目推荐（41 个）

NVIDIA /TransformerEngine

一个用于在 NVIDIA GPU 上加速 Transformer 模型的库，包括在 Hopper、Ada 和 Blackwell GPU 上使用 8 位和 4 位浮点（FP8 和 FP4）精度，以便在训练和推理中提供更好的性能和更低的内存占用。

intel /neural-compressor

SOTA低比特LLM量化（INT 8/FP 8/MXFP 8/INT 4/MXFP 4/NVFP 4）和稀疏性; PyTorch、TensorFlow和ONNX SYS上领先的模型压缩技术

Tencent /AngelSlim

Model compression toolkit engineered for enhanced usability, comprehensiveness, and efficiency.

intel /neural-speed

An innovative library for efficient LLM inference via low-bit quantization

vipshop /ffpa-attn

Kernel Library for Large Headdim Attention (64~1024, BF16/FP8/FP4), 1.5x~15x↑ vs PyTorch SDPA.

julianmb /q38rocm

Qwen 3.8 27B ROCmFP4 on AMD Strix Halo (Ryzen AI Max+ 395). Up to 36 tok/s via MTP Speculation, TurboQuant & Mesa RADV Wave64.

dnv2003 /v100-skinny

Hand-written NVFP4 W4A16 CUDA kernels for Volta

julianmb /halofpx

High-performance Lemonade alternative for AMD Strix Halo & Radeon — latest MoE models (Ling-3.0-Flash, Qwen 3.8 Flash Next, Ornith 1.5), bleeding-edge RDNA 3.5 Wave64/ROCmFP4 kernels, and silicon-tuned profiles.

hanxiao /DeepSeek-V4.1-Flash-1017-FP8-abliterated-4x-RTX-PRO-6000

Abliterated DeepSeek-V4.1-Flash-1017 (FP8 dense, FP4 experts) on 4x RTX PRO 6000 with vLLM-Moet r21, DSpark + FLy verification and top_p 1.0: 463 tok/s single-stream agent decode at 30k-120k context, 1082 tok/s over 8 streams, 1M context, Flex-start GCE VM

xl03-bit /sm120-nvfp4-training

Full NVFP4 training for GeForce RTX 50-series and RTX PRO 6000 Blackwell GPUs (SM120). Built on Infatoshi/nvfp4-sm120, with software stochastic rounding, random Hadamard transforms and six profiles for speed and memory. On one RTX 5060 Ti 16GB, validation loss fell over 2,048 full-parameter FineWeb updates of a 4.06B model.

stlin256 /CUDABurner

An stress and benchmark utility for NVIDIA GPUs. Measures performance across various precisions (FP64, FP32, TF32, FP16, INT8) and monitors real-time vitals like power, temperature, and clock speeds.

RobTand /gridbook

Out-of-tree vLLM plugin and open format spec for NVFP4-CB / FP8-CB product-codebook weights — 2-6 bit-per-weight LLM quantization served on native Blackwell tensor cores.

FP4-is-All-you-Need /Oz-FP4

Base-13 FP4 limbs for Ozaki-scheme DGEMM emulation on FP4 tensor cores

MurrellGroup /Microfloats.jl

Narrow precision floating point types

FP4-is-All-you-Need /BLAS

Exact matrix products on FP4 Tensor Cores: the Adaptive Weight Encoding (AWE) catalog and the BLAS routines built on it.

devopsfortunato /Qwen3.8-Flash-Next-FP4-Local-DGX-Spark

One-command recipe to serve nvidia/Qwen3.8-Flash-Next-FP4-Local on a single NVIDIA DGX Spark with vLLM (OpenAI-compatible API).

animede /diffusers-ltx2_5

该项目在 GitHub 上没有填写描述

NvMayMay /nvfp4-lora-spark

LoRA fine-tune and serve NVFP4 models on one DGX Spark (GB10, 128 GB UMA): text backbones via generic-family onboarding, plus VLMs (vision tower, or LLM+tower jointly via --train-target both) validated end-to-end on Pixtral and Nemotron-Omni. Fused Triton dequant; runtime-LoRA and merge serving.

A-ENTROPY /hipfire-kvmem

hipfire × KVMem —— 把 KVMem 的分页/分层 KV 移植进 hipfire 这个 RDNA 原生 Rust 引擎，让 262144 上下文在 24 GB 显卡上跑起来；agent 负载实测中位 32 tok/s。Paged/tiered KV (KVMem) ported into the RDNA-native Rust engine hipfire.

mukullokhande99 /XR-NPE

Python implementations for multi-precision quantization in computer vision and sensor fusion workloads, targeting the XR-NPE Mixed-Precision SIMD Neural Processing Engine. The code includes visual inertial odometry (VIO), object classification, and eye gaze extraction code in FP4, FP8, Posit4, Posit8, and BF16 formats.