awq 开源项目推荐（115 个）

intel /neural-compressor

SOTA低比特LLM量化（INT 8/FP 8/MXFP 8/INT 4/MXFP 4/NVFP 4）和稀疏性; PyTorch、TensorFlow和ONNX SYS上领先的模型压缩技术

ModelTC /LightCompress

[EMNLP 2024 和 AAAI 2026]一套强大的工具包，用于压缩大型模型，包括大型语言模型（LLM）、VLM和视频生成模型。

ForeverBlue816 /GRACE

[ICML 2026] GRACE: deployable INT4 VLM via quantization-aware distillation.

RadixRootMind /Awesome-Quant-VLA-WAM

Curated quantization research for VLA & WAM models — 180+ papers, a five-category taxonomy, a roadmap, and a companion PTQ + LIBERO toolkit (quant-vla).

Sandermage /sndr_core_engine

SNDR Core Engine（Genesis）——面向 Qwen3.6 + Gemma4 的 vLLM 运行时补丁覆盖层，适用于消费级 NVIDIA （Ampere sm_86,2× A5000/3090）。Qwen3.6-35B-A3B FP8 ~240 tok/s，27B-int4 混合 GDN+Mamba，Gemma4 26B/31B AWQ，256K ctx。321个音色：TurboQuant k8v4 KV，MTP/DFlash 规范解码，FULL cudagraph，混合 GDN。vLLM pin dev424 + 控制中心图形界面。

aivrar /vllm-windows-build

Native Windows vLLM: 0.27.1 Latest; 0.29.0 Python 3.14/CUDA 13.2 release (CPU TorchAudio), plus Python 3.13/CUDA 13.0 prerelease. PyTorch 2.13, FlashAttention/Rust, OpenAI serving, 10 KV formats, Multi-TurboQuant, experimental CPU/NVMe KV offload. No WSL or Docker.

hec-ovi /vllm-awq4-qwen

vLLM Qwen 3.6-27B (AWQ-INT4) + DFlash speculative decoding on AMD Strix Halo (gfx1151 iGPU, 128 GB UMA, ROCm 7.13). 24.8 t/s single-stream, vision, tool calling, 256K context, OpenAI-compatible, Docker. Matches DGX Spark FP8+DFlash+MTP at a third of the cost. No CUDA.

Leonccaa /ReVolta

Stable, high-throughput inference server for multi-GPU NVIDIA V100 (SM70): Qwen3.8 Flash-Next with AWQ / NVFP4, multimodal input, CPU/disk KV offload, and MTP or DFlash2 speculative decoding.

ncoder-ai /VibeVoice-FastAPI

FastAPI 封装了原始 Vibevoice 1.5B 和 7B 型号，支持 AWQ4 量化

hcd233 /Aris-AI-Model-Server

An OpenAI Compatible API which integrates LLM, Embedding and Reranker. 一个集成 LLM、Embedding 和 Reranker 的 OpenAI 兼容 API

alesha-pro /qwen38-flash-next-4x3090

Qwen3.8 Flash Next AWQ INT4 4x3090 v1: 262K, Vision, FP8 QSA KV and CUDA Graphs on 4x RTX 3090

redlinedtm-jpg /vllm-v100-2080ti-recipes

Running large LLMs on pre-Ampere NVIDIA hardware — Tesla V100 (sm_70), RTX 2080 Ti (sm_75), CMP 170HX. Measured benchmarks, vLLM forks, and the hardware side: NVLink on SXM2 carrier boards, driver traps, cooling, used-kit acceptance.

BoundlessWindMoon /minivllm

A light, transparent, and modular inference & quantization engine for studying LLMs.

ShipItAndPray /turboquant

Compress Any LLM Up to 6x in One Command. Unified CLI for GGUF, GPTQ, and AWQ quantization.

AEON-7 /supergemma4-26b-abliterated-multimodal-nvfp4

NVFP4 AWQ Full quantization of SuperGemma4-26B-Abliterated-Multimodal for Blackwell GPUs — pre-built vLLM container + patches included

harleyszhang /harleyszhang.github.io

🧗‍♂️ harleyszhang 的个人博客

nightthink /K100AI-Model-Deploy

K100AI 模型部署实践 —— 海光 K100-AI 上大模型的成功配置线拉起包（当前：Qwen3.8-27B 六条精选线；持续更新）

hanxiao /Qwen3.8-27B-abliterated-AWQ-SGLang-DFlash2-RTX-PRO-6000

Abliterated Qwen3.8-27B in AWQ-INT4 with SGLang and an FP8 DFlash 2 drafter on half an NVIDIA RTX PRO 6000: 124 tok/s, 237k context, 2 slots, vision

duchengyao /big-vllm

本来叫 nano 的，后来发现装不下 Qwen3.5，就改名叫 big 了

xiersweet2648 /sm120-llm-kernels

Hand-written LLM inference CUDA kernels for consumer Blackwell (RTX 5060 Ti / sm_120) — 1.05x vLLM's Marlin at batch-1 decode, with reproducible Nsight Compute evidence for every level.