ai-inference 开源项目推荐（290 个）

bentoml /BentoML

部署 AI 应用和模型的最简便方式——构建模型推理 API、任务队列、LLM 应用、多模型流水线等！

uxlfoundation /scikit-learn-intelex

Extension for Scikit-learn is a seamless way to speed up your Scikit-learn application

uxlfoundation /oneDAL

oneAPI Data Analytics Library (oneDAL)

intel /dffml

使用机器学习的最简单方式。混合搭配底层机器学习库和数据集来源。轻松生成新数据集或修改现有数据集。

Scottcjn /ram-coffers

LLM infrastructure cost reduction via NUMA-aware weight banking: 147 t/s (8.8x stock llama.cpp) on refurbished enterprise POWER8. Self-hosted inference, no cloud APIs. Part of the Proof of Physical AI stack.

IBM /vLLM-Hook

Repo for vLLM Hook, an vLLM plug-in for programming internal states of models deployed on vLLM

team-telnyx /telnyx-code-examples

Production-ready code examples for Telnyx AI Communications Infrastructure — Voice AI, SMS, SIP, and IoT APIs

Scottcjn /llama-cpp-power8

面向 IBM POWER8 的 llama.cpp 优化：vec_perm 非双联合坍缩、PSE 硬件熵、DCBT 预取。主权推理。Proof of Physical AI 栈的一部分。

pixelrhino-ai /vrhino

A self-contained native runtime and model format for local AI video generation.

PMZFX /intel-arc-pro-b70-benchmarks

Benchmark results and performance data for the Intel Arc Pro B70 GPU (Xe2/Battlemage) - LLM inference, video generation, dual-GPU scaling.

Intrafere /MOTO-Autonomous-ASI

该项目在 GitHub 上没有填写描述

blace-ai /blace-ai

Cross-platform c++ sdk & model hub for cross-platform AI inference. Ready-to-deploy models including Segment Anything 3, Depth Anything 2 and Gemma.

prayangshuuu /hummingbird

hummingbird is a lightweight, zero dependency runtime for massive open source Mixture of Experts (MoE) language models. It unifies SSD, RAM, and VRAM into a single intelligent memory hierarchy, enabling inference of models like GPT-OSS 120B, GLM, DeepSeek, Qwen, and more on consumer hardware

anseryuer /Local_LLM_Deployment_Guide_Chinese

本地部署大语言模型的中文教学

xflops /flame

A distributed engine for elastic workload, e.g. AI Agent, RL, Quant

Quazmoz /InferBridge

Windows-first OpenAI-compatible local LLM server powered by OpenVINO GenAI for Intel CPU/GPU/NPU, with chat UI, model conversion, and setup scripts.

okba14 /NeuroHTTP

High-performance, AI-native server built from scratch in C + hand-written Assembly (SSE2/AVX2/AVX-512). Supports 10 AI providers — OpenAI, Groq, Anthropic Claude, Google Gemini, DeepSeek, Moonshot/Kimi, Zhipu GLM, Perplexity, Mistral AI, and local models via Ollama — all in one binary.

jaeseok614 /ai-hardware-fit

Can my GPU run this? Pick a GPU to get 3 fitting AI models, quantization, VRAM, speed ranges, and run commands. 한국어·English.

Josia-doit /ComfyUI_JosiaNodes

一套功能完整、轻量稳定的 ComfyUI 实用节点集。初衷是为了平替一些断更插件的功能，会持续更新一些新开发的节点插件。有好的创意可以发Issues，我会力所能及想办法做出来。全中文界面，原生控件，不依赖第三方库。

microsoft /Build26-LAB520-get-started-with-models-in-microsoft-foundry-to-build-ai-apps

该项目在 GitHub 上没有填写描述