GPT-QModel ⚡
An extensible platform for LLM quantization, validation, and deployment.
Compress, evaluate, and run models across supported GPUs, accelerators, and CPUs.
Documentation ·
Quantization guide ·
Reference index ·
Releases
Latest News 🗞️🚀
7.11.4
- Quantization: SmoothMSE keeps the unclipped range when clipping would increase error, and QQQ’s GSQ calibration now uses the same FP32 activation scales as inference. AWQ samples routed-expert activations uniformly across the calibration set instead of favoring later batches. Paro retains mixed-shape batches, and SenseNova-Loop calibrates on every pass through its repeated layers.
- Quantization memory: AWQ and Paro limit how many activation rows they retain. AWQ reuses its expert sampling buffers and scales experts in place across GPUs, while EXL3 spreads expert Hessians across the configured GPU pool. ROCm allocator defaults reduce peak reserved VRAM for wide MACaQ layers.
- Inference: Marlin speeds up large-batch prefill and fixes GPTQ/AWQ output errors under GPU contention and MXFP8/MXFP4 execution. Portable QQQ and VPTQ Torch kernels now support compiled generation. Fixed crashes when loading another model during compiled inference and improved EXL3 BF16 execution and Paro dispatch for long prefills.
- Compat: Fixed Transformers
device_map="auto" when GPTQ, RTN, or AWQ layers spill to CPU or disk. Checkpoint loads preserve required FP32 tensors, and re-saves preserve quantized embeddings and output heads. Marlin now requires NVIDIA Ampere or newer.
7.11.2
- Inference:
optimize() now compiles the forward pass used by generation, and deleting compiled models releases VRAM. Supported CPU kernels compile with Inductor without graph breaks. Torch compile fallback covers the first-use compile lock and dynamo failure classification, and the generation compile policy explains when static-cache auto-compile is suppressed. EXL3 uses native kernels for BF16, and Paro selects Triton on ROCm.
- Models: Added quantization for SenseNova-Loop 9B and Decision Transformer, which calibrates on trajectories without a tokenizer. Fixed checkpoint round trips for Qwen2.5-Omni, Qwen3-Omni, and InternVL.
- Quantization: GPTQ and EXL3 use less GPU memory. NF12 and MACaQ can resume from checkpoints, and models stay runnable after failed or cancelled runs.
- Calibration: Fixed token positions and masks in padded and multimodal batches. Chat datasets retain tool and document context, and AWQ still scales MoE experts that received tokens when others received none.
- GGUF: Added ggml 1.0 support, including reading and dequantizing FP8 E4M3 tensors. The
gguf extra requires llama-cpp-python>=0.3.37.
- Compat: Fixed GPTQ and AWQ CPU offload in Transformers and added CPU readers for AWQ GEMV_FAST and LLM_AWQ. ParoQuant is now Paro; existing checkpoints still load.
API note: Use CalibrationConfig for calibration options. GPTQModel.export() and public MLX-LM conversion were removed.
7.11.0
- MACaQ: Native MXFP4 W4A4 quantization for low-precision weights and activations, with CUDA, ROCm, and Apple silicon inference. Includes packed checkpoints and a portable Torch fallback.
- NF12: Lossless 12-bit compression of BF16 weights reduces source-weight storage during calibration without changing the restored values. Packed inference now includes native Metal support alongside CUDA, HIP, and Torch paths. MLX · MoE capture contract
- Tenstorrent: Packed GPTQ inference through Torch/XLA and TTNN, multi-card execution, and block-float quantization. Long-context Llama serving adds paged caching and chunked prefill; compatible block-float checkpoints also run on CUDA, ROCm, and CPU. Guide · Serving · Device ownership
- Quantization: Resumable distributed EXL3 jobs (projection publication recovery), broader mixed-bit allocation across AWQ/EXL3/GGUF, and SmoothQuant preprocessing improve large-model workflows and quality control. Quantizer calibration flows document the shared Hessian-failure and zero-sample fallback contracts. MoE routing override and bypass defines when each mode is honoured and when it fails closed, and its AWQ bypass replay kwargs contract keeps instance-level replay wrappers from widening a standard MoE block’s signature. Shared-input Hessian dedup covers QQQ/VPTQ/MACaQ/EXL3 and GPTAQ/FOEM, and the
desc_act sink-row cap keeps its row-norm bound on device. Optional calibration attention-sink exclusion drops BOS-token rows from the calibration statistics. Embedding-endpoint quantization is requantize-only on a loaded checkpoint; a fresh quantize() refuses the unsupported modes before calibration.
- Inference: Opt-in attention and MLP projection fusion, faster quantized embedding lookups, and improved small-batch inference. Native backend selection and packed-weight storage are preserved. Fusion guide
- Configuration: Typed backend configurations unify backend selection and tuning, with explicit numerical and worker controls instead of environment switches.
- Calibration: Broader dataset input support, document-isolated AWQ replay, and opt-in attention-sink exclusion improve calibration coverage and control. Calibration layer replay documents the per-definition replay contract, including PanGu-α’s top-query layer.
- Memory: Reduced packed-weight duplication, bounded EXL3 scratch, and shorter-lived inference workspaces lower memory use. Fused-expert MoE loads keep an excluded GPU unclaimed during Defuser conversion, and AWQ folds each expert on its own device in a balanced MoE pool., and compiled forwards stay on the model’s device through compiled forward device scope.
- Models: Added OLMo Hybrid, Granite SWA/MoE SWA, and Dots3 Note quantization, plus broader decoder-weight coverage and Transformers 5.22 compatibility. Model notes · Decision Transformer calibration
- Compat: Major fixes across dense/MoE calibration, multimodal replay, checkpoint round trips, Transformers model loading, multi-GPU memory use, and ROCm execution. Unsupported combinations fail clearly rather than loading incomplete weights. Full notes · Cancelled-run model state
- Security: Runtime checks for vulnerable PyTorch versions, explicit checkpoint-code trust, stronger checkpoint validation, and bounded server/media requests. Runtime safety · Serving security
| Recent release |
Highlights |
| 7.10.0 |
Ricursive Ripple support, faster decode, checkpoint resharding, and multi-GPU fixes. |
| 7.9.0 |
Kernel tuning, MLX improvements, new models, and numerical fixes. |
| 7.8.0 |
GSQ/RCO training and allocation, expanded model support, and compatibility fixes. |
Detailed release notes · Older changelog
GPT-QModel brings calibration, quantization, evaluation, conversion, and accelerated
inference into one workflow. Start from a pretrained model, choose a compression
recipe, validate its quality, and save a checkpoint for native inference or a
compatible deployment engine.
Methods and compression 🎛️
Quantized kernels keep only packed weights resident; per-call QQQ and EXL3 kernel scratch, one quantization stream and one tile-scratch pair per device, per-device plan lifetimes and the single GGUF CUDA weight copy are documented in kernel scratch lifecycle. CPU layer packing quantizes and packs each bounded input block in one shift/reduce pass; see CPU block packing.
Control, scale, and quality ⚙️
- Choose where to spend precision. Mix bit widths, keep sensitive modules
unquantized, and use activation-quantization or preprocessing policies where
supported. Quantization protocols let you organize multi-stage recipes.
- Select embedding endpoints explicitly.
QuantizeEmbedConfig rejects an
unknown embed_quant_mode or a non-bool embed_only before any layer
replay instead of silently quantizing nothing; see
embedding quantization config validation.
- Keep floating activation searches finite. NVFP4 block-scale ranking
normalizes candidate error by an exact power of two so large finite
activations cannot overflow the comparison and silently return zeros; see
NVFP4 candidate ranking.
- Fit larger models. Multi-GPU calibration, CPU/disk offload, and checkpointed
execution help manage memory and recover interrupted jobs. Disk-offload run
directories
are per-run scratch space and are removed at teardown or failure, and the
in-session model
is fully materialized when
quantize() returns;
reference offload
keeps untouched base modules as source-checkpoint references instead of
second copies, and releases pinned staging memory. Quantized MoE reloads park
fused expert projections on meta until the packed weights are read
(meta fused-expert shell).
Distributed EXL3 workflows support durable replay and resume.
- Recover accuracy. Use GSQ refinement, EoRA low-rank compensation, rotation,
smoothing, and norm tuning for supported recipes. LoRA training and adapter
loading support further task adaptation; exported EoRA
rank_pattern keys follow the adapter rank-pattern contract so PEFT builds the trained rank per module. Rotation model support lists the exact definitions that opt in. AWQ GSQ uses the
packed-code scoring contract to evaluate exported candidates.
QQQ inference and GSQ calibration share one activation INT8 grid contract. The fused relaxation kernel keeps the eager precision contract
for BF16 and FP32 logits, and per-linear refinement searches the exported
codes and refits scales under the scalar refinement
contract.
- Trust the fold. AWQ divides a scale only into the norm that actually feeds
a quantized block, including multimodal wrappers and hybrid layer shapes; see
the AWQ scaling groups contract. Scale search keeps
routing in the original-input coordinates, validates replay coverage, and
restores temporary weights on failure; see search safety.
- Reject unrepresentable AWQ pack scales. The native GEMM/GEMV packers
validate the actual FP16 storage cast before replacing module state; see
AWQ pack-time scale representability.
- Decode AWQ weights in range. The portable Torch AWQ forward picks its
decode dtype from the stored scale range, so a wide or tiny checkpoint scale
no longer becomes
inf/0 before the product; see
AWQ Torch forward decode range.
- Keep the AWQ extraction table out of a capture. A four-bit dequant that
first runs inside
torch.cuda.graph(...) records its offset table instead of
computing it, so the table is published only from eager calls; an abandoned
capture can no longer leave later eager dequant decoding with zeros. See
capture-safe extraction offsets.
- Keep the AWQ bias out of the scale dtype. A non-quantized bias is added
after the reduction, so packing keeps its source precision instead of casting
it to FP16; see
AWQ pack-time bias storage.
- Keep the Marlin bias in the checkpoint. The native Marlin kernel adds the
bias to its FP32 accumulator, so the buffer keeps its checkpoint width and
order and the padded, permuted operand is derived per launch; see
Marlin bias epilogue and storage.
- Measure before deploying. Evaluate perplexity and downstream tasks, compare
against a non-quantized baseline, and test prompts representative of your users.
Compression settings trade model size, calibration cost, speed, and quality.
- Preserve causal replay. Bloom AWQ keeps captured attention context and
rejects unsafe residual folds; see Bloom replay safety.
- Capture masks a model passes positionally. GPT-2 hands its causal mask to
each block as a positional argument; reading only keyword arguments dropped it
and replayed every layer without causality. See
positional replay mask capture.
- Classify additive masks before dropping padding. An all-zero extended mask
means “no padding”, not “all padding”; layer replay, user calibration and AWQ
share one keep-mask classifier. See
attention-mask keep classification.
- Replay each calibration document separately. AWQ token-row replay rebuilds
a block-diagonal per-document mask and keeps each row’s original position;
additive masks treat
0 as attended. See
AWQ calibration aggregation.
- Resolve the rotary label from the attention. AWQ scale search rebuilds
position_embeddings with the attention’s own rope label and keeps a
per-rope-type mapping intact; see
AWQ rotary replay selection.
- Trim padding before the position budget. Over-long calibration rows are
measured on their real tokens, not their padded width, so a left-padded row
cannot lose real tokens to its own padding. See
calibration row trimming.
- Pack before filtering when concatenating. With
concat_size set, the
per-row minimum-length filter and position trim run on the packed windows
instead, so short documents are combined rather than discarded and a long
document is not cut before it is windowed. See
calibration row trimming.
- Keep VLM calibration rows intact. Pre-tokenized rows (
input_ids) take the
base text path instead of a chat template, columnar mappings are unpacked, and
conversations reject CalibrationConfig.concat_size. See
VLM calibration input normalization.
Inference and deployment 🚢
Run compatible checkpoints directly with GPT-QModel, integrate with Transformers
(import gptqmodel first so packed MoE experts load through the
native bridge),
or deploy supported GPTQ/AWQ formats through vLLM and SGLang. Apple silicon has
native MLX quantization and packed inference. Optional projection fusion can
combine supported attention and MLP operations without changing the saved format.
EoRA/LoRA adapters, mixed-precision checkpoints, and external engines have their
own compatibility limits; support for a method does not imply support in every engine.
Start with the quantization guide.
External runtimes support only specific method/format combinations; check the
integration and backend tables before exporting.
| Platform |
Hardware |
Runtime overview |
| Linux |
NVIDIA GPUs |
CUDA kernels; Turing or newer, with backend-specific minimums |
| Linux |
AMD GPUs |
ROCm 10.1+; architecture-specific Torch/Triton/HIP paths |
| Linux |
Tenstorrent |
Torch/XLA/PJRT and native TTNN |
| Linux |
Huawei Ascend |
Native Torch through torch-npu / CANN |
| Linux |
Intel XPU |
Torch and TorchFused |
| Linux |
Intel/AMD CPUs |
Torch and optimized x86 kernels |
| macOS |
Apple silicon |
Native MLX/Metal quantization and inference; MPS/Torch |
| Windows 11 |
NVIDIA GPUs / CPU |
Torch |
Support depends on the chosen method, backend, dtype, and device architecture.
See the full hardware and kernel matrix,
ROCm guide, Tenstorrent guide,
and Apple silicon guide.
Tenstorrent-compatible block-float formats also run on CPU without Tenstorrent
hardware. See the CPU block-float guide.
Model Support 🤖
Support spans dense, mixture-of-experts (MoE), vision-language, and audio-language
families. The compact list below keeps model coverage visible; the
complete registry lists exact model-type aliases and
model-specific limitations.
| Families |
Supported models and variants |
| A |
A.X-K2; AfMoE / Trinity; Agnes 3.0 Flash; Apertus 1 / 1.5 |
| B–C |
Baichuan; Bailing MoE / Hybrid (LING / RING); Bloom; Brumby; ChatGLM; CodeGen; Cohere 1 / 2 / 2 MoE / Compass (North Mini / Micro Vision) |
| D |
DBRX / DBRX Converted; DeciLM; DeepSeek V2 / V3 / V3.2 / V4 / VL / VL2 / OCR2; DiffusionGemma; Dots1 / Dots3 Note; Dream |
| E–F |
ERNIE 4.5 / MoE / VL MoE; EXAONE 3 / 4 / 4.5; Falcon / Falcon H1 / Falcon Mamba; FastVLM / LLaVA-Qwen2 |
| G |
Gemma 1–4 / 3n / Unified; GLM / GLM4 / GLM4V / GLM5 / OCR / ASR; GPT-2; GPT BigCode; GPT-Neo / GPT-NeoX; GPT-OSS; GPT-J; Granite / Granite SWA / MoE / MoE SWA / MoE Hybrid; GRIN-MoE |
| H |
Helium; HRM; Hunyuan V1 / VL / OCR; HY-V3; Hymba |
| I–J |
Inkling; Instella; Intern S1 / S2 Preview; InternLM 1 / 2 / 2.5; InternVL 3 / InternVL Chat; Jamba / Jamba2 |
| K |
K2-Horizon (Dense / MoVA); Kimi K2 / K2.5 / K3 / Linear; Klear; KRAFTON Raon-1 mini |
| L |
Laguna; LFM2 / LFM2 MoE / LFM2-VL; LFM3 / LFM3 Omni / LFM3-VL; LLaDA2 MoE; Llama 1–4 / TinyLlama / Nemotron Ultra; Llama 3.2 VL; LLaVA-1.5; LocateAnything; LongCat Flash; LongLLaMA |
| M |
Mage-VL; Marin; MiMo / MiMo V2 / MiMo V2 Flash; MiniCPM / MiniCPM3 / MiniCPM-O / MiniCPM-V; MiniMax M2 / M3-VL; Mistral / Mistral3 / Ministral3; Mixtral; MobileLLM; MOSS; MPT; Muse Glimmer |
| N |
Nanbeige 4.2; Nemotron NAS / H / H Puzzle / Omni / Labs Diffusion |
| O |
OLMo / OLMo 2 / 3 / Hybrid / OLMoE; OpenVLA; OPT; Ouro; Ovis 1.6 / 2 / 2.5 / 2.6 MoE / 2.6 Next |
| P |
PanGu-α; Phi 1–4 / Phi MoE |
| Q |
Qwen 1–4 / 3.5 / 3.6 / 3.8 / MoE / Next; Qwen 2 / 2.5 / 3 VL / 3 VL MoE; Qwen 2.5 / 3 Omni; Qwen2-Audio; Qwen-Drive 1.0 |
| R |
RecursiveSI Flint; RefinedWeb; Ricursive Ripple |
| S |
Seed-OSS; SmolLM3; SmolVLM; Solar Open / Open 2; Spark 2.5; StableLM; StarCoder2 |
| T–V |
TeleChat2; Unlimited-OCR; Voxtral |
| X–Z |
Xing4.0 (29B-A4B); XVERSE; Yi; Zamba / Zamba2; ZDTaichu 5.0 |
For multimodal checkpoints, support may target the language decoder rather than
every vision/audio component. Kimi K3 / Linear requires the dedicated native-source
calibration workflow; Qwen 3.5 requires Transformers 5.2.0 or newer. Check the
model notes before starting a large run.
Browse quantized models on Hugging Face
Install 💾
Use a Python environment with a compatible PyTorch build for your hardware:
pip install gptqmodel
# Or: uv pip install gptqmodel
PyTorch 2.13.x or >= 2.15.0 is required; known-vulnerable versions are
blocked by the runtime safety check.
Compiled package backends build and cache extensions on first use (JIT),
so install the compiler and CUDA/ROCm toolkit required by your backend. HIP
builds stage their sources under the cache; see
HIP JIT source staging.
See installation and optional extras
for source installs and platform setup. Install vLLM or SGLang in a separate
environment aligned with that engine’s dependency pins.
The native GGUF C++ backends live in the gguf extra
(llama-cpp-python>=0.3.37, which vendors ggml 1.0). GGUF_CPP reads the
loaded release through ggml_version(); ggml 1.0 adds the F8_E4M3 tensor
type, which the native GGUF reader can dequantize.
Quick start 🚀
Inference 💬
Load an existing quantized checkpoint and generate text. GPT-QModel selects a
compatible installed backend for the checkpoint and your device; you can choose
an explicit backend when tuning a deployment.
from gptqmodel import GPTQModel
model = GPTQModel.load("ModelCloud/Llama-3.2-1B-Instruct-gptqmodel-4bit-vortex-v2.5")
tokens = model.generate("Uncovering deep insights begins with", max_new_tokens=64)[0]
print(model.tokenizer.decode(tokens))
Loading and inference
covers backend selection, generation, and adapters. Only enable
trust_remote_code=True for checkpoint code you trust. For model types the
installed Transformers implements natively, the native class is used unless the
checkpoint declares a remote-only layout; see
docs/trust-remote-code-isolation.md.
Generation settings preserved during quantized checkpoint re-save are described
in docs/generation-config-round-trip.md.
Re-saving a loaded fused-expert MoE defuses the export shell and fails closed on
dropped quantization payloads; see
docs/quantized-moe-resave-defuse.md.
export(..., "hf") picks its dense decoder from the checkpoint’s declared
quant_method and refuses methods that have none (Paro, MACaQ, GGUF, QQQ, NF12,
VPTQ, EXL3) instead of decoding them with another layout’s decoder; see
docs/export-hf-format-detection.md.
In-session QQQ kernel rehoming and cache initialization are documented in
docs/qqq-kernel-selection.md.
QQQ’s shared-memory staging of the per-group scales and the reduction-scratch
ordering are documented in
docs/qqq-group-scale-staging.md.
Quantized modules load their scales and other quantization buffers at the dtype
the checkpoint stores, whatever model dtype you request; see
checkpoint buffer dtype.
Quantization 🗜️
This example creates a 4-bit GPTQ checkpoint using 256 streamed C4 samples.
Use calibration text representative of your workload; the dataset and precision
settings affect the quality of the resulting model.
from datasets import load_dataset
from gptqmodel import CalibrationConfig, GPTQConfig, GPTQModel
dataset = load_dataset(
"allenai/c4",
data_files="en/c4-train.00001-of-01024.json.gz",
split="train",
streaming=True,
)
calibration = [row["text"] for row in dataset.take(256)]
model = GPTQModel.load(
"Qwen/Qwen3-0.6B", GPTQConfig(bits=4, group_size=128),
)
model.quantize(CalibrationConfig(dataset=calibration, batch_size=1))
model.save("Qwen3-0.6B-GPTQ-4bit")
quantize() takes every calibration choice in one CalibrationConfig: dataset,
concat_size, sort ("desc"), batch_size (1), min_length (10),
concat_separator, exclude_attention_sinks (False) and the EoRA
adapter_dataset. See
the calibration API.
The end-to-end guide covers other methods,
calibration, saving, reloading, and validation.
For Apple silicon, use the native MLX workflow.
What to tune next 🔧
- Quality versus size: Start from a supported recipe, then compare bit widths
and group sizes. Use mixed precision or exclusions for sensitive layers rather
than assuming one configuration suits the whole model.
- Calibration: Choose representative text and enough examples to cover your
use case. MoE and multimodal models may need specialized data preparation.
- Memory and recovery: Configure multi-GPU placement and offload before a
large run, and use checkpoint/resume where supported. A CPU-offloaded
placement also changes the autograd context
generate() uses; see
generate on a CPU-offloaded model.
- Deployment: Reload the saved checkpoint, evaluate it, and verify its format
in the intended engine. A successful native load is not an external-engine
compatibility guarantee.
See the configuration reference
and quantization recipes for the available controls.
Guides and reference 🧭
Kernel contracts, implementation flows, and focused technical notes are in the
documentation index, not repeated on this page.
Quality: GPTQ 4-bit evaluation 🏆
Quantization quality depends on the source model, recipe, and calibration data.
The Vortex collection provides GPTQ examples; the charts below illustrate selected
evaluations, not a guarantee for every supported model or method.
ModelCloud quantized Vortex models

Organizations and projects using GPT-QModel 🌐
Selected public references include Hugging Face, Intel, NVIDIA, Alibaba Cloud,
Fujitsu, and IBM Research.
This is not an exhaustive customer list.
Contributions welcome 🛠️
We welcome all contributors, including people working with human-guided AI
agents. Code, model and hardware support, documentation, bug reports, reviews,
and testing all help move GPT-QModel forward. See the
developer guide
to get started.
Support ongoing development 💛
Supporting a wide range of hardware and models requires continuous CI testing,
accuracy validation, and compatibility work. That takes substantial compute and
AI-agent tokens. Contributions of compute, model/API credits, or financial
support of any size help sustain project maintenance and future development.
Interested in sponsoring? Message Qubitium on X.
If you find GPT-QModel useful, sending even a couple of dollars through X is
also appreciated. Ongoing sponsors will be credited on this README page.
License 📜
GPT-QModel is licensed under Apache-2.0. The optional, JIT-loaded
Swordfish kernel is AGPL-3.0-or-later.
Attributions and citations 📚
GPT-QModel builds on the research and open-source projects below. Please cite
GPT-QModel and the methods you use; full author lists, BibTeX, and attribution
details are available in the citation reference.
Cite GPT-QModel (BibTeX)
@misc{qubitium2024gptqmodel,
author = {ModelCloud.ai and [email protected]},
title = {GPT-QModel},
year = {2024},
howpublished = {\url{https://github.com/ModelCloud/GPTQModel}}
}
| Work |
Authors / project |
Paper or reference |
| GPT-QModel |
ModelCloud.ai and contributors |
Project |
| GPTQ |
Elias Frantar et al., IST-DASLab |
Paper |
| AWQ |
Ji Lin et al. |
Paper |
| Paro |
Yesheng Liang et al., Z-Lab |
Paper · Code |
| MACaQ |
Lorenz K. Muller et al., Huawei CSL |
Paper · Code |
| QQQ |
Ying Zhang et al., Meituan |
Paper |
| VPTQ |
Yifei Liu et al., Microsoft |
Paper · Code |
| GGUF / llama.cpp |
Georgi Gerganov and ggml-org contributors |
Project |
| ExLlama V3 / EXL3 |
turboderp and contributors |
Project |
| NF12 / NyoomFloat12 |
Liberman et al., Together AI |
Research and codec |
| EoRA |
Shih-Yang Liu et al., NVIDIA |
Paper |
| GSQ |
Alireza Dadgarnia et al., IST-DASLab |
Paper · Code |
| RCO |
Michael Helcig and Dan Alistarh, IST-DASLab |
Paper · Code |
| GPTAQ |
Yuhang Li et al., Yale Intelligent Computing Lab |
Paper |
| GAR |
T. Gafni, A. Karnieli, and Y. Hanani, Intel |
Paper |
| FOEM |
Xingyu Zheng et al. |
Paper |
| Norm Tweaking |
Liang Li, Qingyuan Li, Bo Zhang, and Xiangxiang Chu |
Paper |
| Fast Hadamard Transform |
Tri Dao, Dao-AILab |
Code |
| Marlin |
Elias Frantar et al. |
Paper |
| Machete |
vLLM contributors |
Code |
| Swordfish |
AlpinDale |
Article |
评论