fp8 开源项目推荐（194 个）

NVIDIA /TransformerEngine

一个用于在 NVIDIA GPU 上加速 Transformer 模型的库，包括在 Hopper、Ada 和 Blackwell GPU 上使用 8 位和 4 位浮点（FP8 和 FP4）精度，以便在训练和推理中提供更好的性能和更低的内存占用。

NVIDIA /cudnn-frontend

cuDNN 前端是 NVIDIA 现代开源的 cuDNN 库入口，以及一个不断增长的高性能开源内核集合。

Azure /MS-AMP

Microsoft Automatic Mixed Precision Library

intel /neural-speed

An innovative library for efficient LLM inference via low-bit quantization

vipshop /ffpa-attn

Kernel Library for Large Headdim Attention (64~1024, BF16/FP8/FP4), 1.5x~15x↑ vs PyTorch SDPA.

aredden /flux-fp8-api

Flux diffusion model implementation using quantized fp8 matmul & remaining layers use faster half precision accumulate, which is ~2x faster on consumer devices.

jpezzulli /sglang-rtxpro6000

Optimized SGLang runtime for Qwen3.8-27B FP8 with DFlash2 and Qwen3.8 Flash-Next NVFP4 with FR-Spec on one NVIDIA RTX PRO 6000 Blackwell 96 GB GPU (SM120): 524K context, HiCache and NIXL.

DominikBucko /qwen38-flash-next-2x3090

Qwen3.8-Flash-Next on 2× RTX 3090: with 128 GB RAM up to 4,191 tok/s prefill · 111.5 tok/s decode (131K prompt), full 256K window at 2,865 tok/s; with 64 GB RAM 3,410 tok/s prefill · 84 tok/s decode. New: opt-in uncensored mode (runtime abliteration, no new weights).

joeynyc /MiniMax-H3-DGX-Spark

Measured SM121 compatibility recipe for MiniMax H3 FL2VA on one NVIDIA DGX Spark with vLLM-Omni and online FP8.

astding /mmaemu

Bit-exact emulation of NVIDIA tensor-core FP8 GEMMs (torch._scaled_mm, Transformer Engine) as an H100 PCIe or RTX 4060 computes them, on the CPU or another CUDA GPU

0xBakeer /Qwen3.8-27B-FP8-on-a-single-DGX-Spark

Serving Qwen3.8-27B-FP8 on a single DGX Spark (GB10): 7.88 to 58.5 tok/s single-stream from decode strategy alone, weights untouched. Speculative decoding and prefix caching benchmarked, plus DFlash 2 — the only Qwen3.8-27B build that can serve it under vLLM.

hanxiao /DeepSeek-V4.1-Flash-1017-FP8-abliterated-4x-RTX-PRO-6000

Abliterated DeepSeek-V4.1-Flash-1017 (FP8 dense, FP4 experts) on 4x RTX PRO 6000 with vLLM-Moet r21, DSpark + FLy verification and top_p 1.0: 463 tok/s single-stream agent decode at 30k-120k context, 1082 tok/s over 8 streams, 1M context, Flex-start GCE VM

halt95 /merlin-engine

vLLM 0.33 fork for consumer GPUs: 3 x 262K sessions on 4x RTX 3090, Qwen 3.8 Flash-Next at 207 tok/s, up to 17 on bigger cards. Seven checkpoints incl. Qwen3.8-27B, Nemotron 3.5 Super VL (1M) and Ornith 1.5. One command: merlin serve.

zerfoo /zerfoo

Pure Go machine learning framework. Train, run, and serve ML models with go build. Zero CGo.

MerkyorLynn /lynn-engine

Lynn 原生 LLM 推理引擎 · W4A8/NVFP4 量化 · 自写 CUDA/Triton kernel · MoE · 投机解码 | Lynn-native LLM inference engine for NVIDIA Blackwell

tashiscool /fp8-mps-metal

FP8 Metal compute kernels for Apple Silicon MPS — fixing what PyTorch doesn't support yet. FLUX/SD3.5/ComfyUI on Mac.

KumphanartDansiri /v100-vllm-2026

Benchmarks and notes for running modern LLMs with vLLM on 8x Tesla V100-32GB in 2026.

maeddesg /vulkanforge

LLM inference engine for AMD RDNA4 — Rust + Vulkan compute shaders, gguf & native FP8.

spinfinc /spinf-serve

Bulk classification and decision inference on one GPU: the content is read once, every question about it is scored in the same pass. OpenAI-compatible, Apache 2.0

graphcore-research /jax-scalify

JAX Scalify: end-to-end scaled arithmetics