gptq 开源项目推荐（119 个）

intel /neural-compressor

SOTA低比特LLM量化（INT 8/FP 8/MXFP 8/INT 4/MXFP 4/NVFP 4）和稀疏性; PyTorch、TensorFlow和ONNX SYS上领先的模型压缩技术

ModelCloud /GPTQModel

LLM model quantization (compression) toolkit with HW acceleration support for Nvidia, AMD, Intel GPU and Intel/AMD/Apple CPU via HF, vLLM, and SGLang.

shm007g /LLaMA-Cult-and-More

Large Language Models for All, 🦙 Cult and More, Stay in touch !

bobazooba /xllm

🦖 X—LLM：前沿与简易LLM精炼

sergqwer /strata-nvfp4

Strata fork: Qwen3.8-Flash-Next 125B MoE in NVFP4 on one RTX 20-50 card (12 GB+) and 64 GB of RAM or more. Setup installs our GPTQ quants (huihui abliterated, OrcaRouter) from Hugging Face. CPU experts on AVX-512 or AVX2 (+AVX-VNNI), all experts in RAM from 92 GB, a low-RAM mode below, vision, W4A8 prompts. Windows / CUDA 13.

1b5d /llm-api

Run any Large Language Model behind a unified API

chenhunghan /ialacol

🪶 Lightweight OpenAI drop-in replacement for Kubernetes

taishan1994 /LLM-Quantization

记录量化LLM中的总结。

abhinand5 /gptq_for_langchain

A guide about how to use GPTQ models with langchain

WestWaters /pollard-weights

A tiered-memory system design for workloads that don't fit in RAM: measure the working set, pin the hot tier, stream the cold tier from flash. Ships the residency calculator, measurement harnesses, and the build recipes behind it. Predictions validated against public benchmarks.

seyf1elislam /OneClick_LLM_API_onColab

Run gguf LLM models in Latest Version TextGen-webui and koboldcpp

tripathiarpan20 /self-improvement-4all

Private self-improvement coaching with open-source LLMs

hcd233 /Aris-AI-Model-Server

An OpenAI Compatible API which integrates LLM, Embedding and Reranker. 一个集成 LLM、Embedding 和 Reranker 的 OpenAI 兼容 API

chinoll /chatsakura

ChatSakura：Open-source multilingual conversational model.（开源多语言对话大模型）

groxaxo /GPTQ-Pro

GPTQ optimized for Ampere (RTX 3090/3060) — Marlin JIT nvcc compatibility fixes included

ShipItAndPray /turboquant

Compress Any LLM Up to 6x in One Command. Unified CLI for GGUF, GPTQ, and AWQ quantization.

matlok-ai /bampe-weights

This repository is for profiling, extracting, visualizing and reusing generative AI weights to hopefully build more accurate AI models and audit/scan weights at rest to identify knowledge domains for risk(s).

Aqirito /A.L.I.C.E

A.L.I.C.E (Artificial Labile Intelligence Cybernated Existence). A REST API of A.I companion for creating more complex system

Abeshith /FineTuning_LanguageModels

🎯 Fine-tune large language models and use them for text-related tasks. This repository provides a straightforward approach to fine-tuning models like Gemma, Llama 🦙, and Mistral 🌪️ for various NLP tasks. 🔧 It includes training 📚, fine-tuning 🛠️, and inference pipelines ⚙️. 🚀

duchengyao /big-vllm

本来叫 nano 的，后来发现装不下 Qwen3.5，就改名叫 big 了