FlashInfer: Kernel Library for LLM Serving
- Updated
Nov 1, 2026 - Cuda
distributed-inference 开源项目推荐（195 个）
相关标签
A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.
cuda inference openai llama maas rocm ascend llm llm-serving vllm genai llm-inference qwen deepseek sglang distributed-inference high-performance-inference mindie
- Updated
Oct 30, 2026 - Python
State of the art inference performance with modern accelerators on Kubernetes
- Updated
Oct 31, 2026 - Python
High Performance Serving Runtime For Interactive Super Intelligence System
text-to-speech streaming interactive cuda inference pytorch tts speech-recognition music-generation asr model-serving multimodal openai-api audio-generation sglang distributed-inference
- Updated
Oct 31, 2026 - Python
Peer-to-peer LLM inference in the browser: pool your devices to run big open models, for chat and coding agents.
browser webrtc peer-to-peer inference webgpu wgsl llm local-llm speculative-decoding distributed-inference
- Updated
Oct 16, 2026 - JavaScript
Best practices, reference architectures, and examples for distributed AI training and inference on AWS.
kubernetes aws gpu slurm efa distributed-training eks parallelcluster hyperpod generative-ai distributed-inference physical-ai
- Updated
Oct 31, 2026 - Shell
Run large AI models across your computers and servers. Powered by Colibri. Pure C.
c peer-to-peer self-hosted colibri cluster-computing mixture-of-experts llm-inference local-ai distributed-inference
- Updated
Oct 31, 2026 - C
[Official] Prima.cpp: Scale Your Local AI Beyond One Device.
- Updated
Sep 6, 2026 - C++
Split FLUX.2 and LTX 2.3 across two GPUs (LAN or same-machine) — NVENC compresses activations live on the wire. Icarus (ComfyUI node) + Daedalus (back-half server).
flux pytorch image-generation multi-gpu nvenc video-generation pipeline-parallelism diffusion-models flux2 comfyui distributed-inference comfyui-node ltx-video
- Updated
May 17, 2026 - Python
一個基於 llama.cpp 的分佈式 LLM 推理程式，讓您能夠利用區域網路內的多台電腦協同進行大型語言模型的分佈式推理，使用 Electron 的製作跨平台桌面應用程式操作 UI。
- Updated
Aug 23, 2026 - JavaScript
Educational distributed training and inference library for heterogeneous local hardware: Mac minis, Raspberry Pis and GPUs over plain Python sockets. FSDP, GRPO, parameter-server, expert and data parallelism.
raspberry-pi distributed-computing pytorch educational zero socket-programming data-parallelism parameter-server edge-computing model-parallelism distributed-training heterogeneous-computing mac-mini apple-silicon fsdp llm-training local-ai distributed-inference grpo expert-parallelism
- Updated
Oct 7, 2026 - Python
Free, open-source app that runs AI chat models on your own computer — and lets computers team up over the internet to run models too big for one machine. No account, no crypto. OpenAI- and Anthropic-compatible API.
rust ai mcp decentralized p2p inference self-hosted candle libp2p huggingface openai-api llm chatgpt-alternative local-llm gguf anthropic-api distributed-inference claude-code ollama-alternative openclaw
- Updated
Nov 1, 2026 - Rust
Pool RAM across old PCs, Macs, and Android devices into a unified local AI cluster
- Updated
Oct 26, 2026 - Python
Distributed image and video generation in ComfyUI for two NVIDIA DGX Sparks.
nvidia image-generation video-generation diffusion-models comfyui distributed-inference gb10 dgx-spark
- Updated
Oct 31, 2026 - Python
Run any model on Intel silicon
- Updated
Oct 31, 2026 - Rust
A chatbot, with a large language model split across its users, entirely in the browser.
- Updated
Oct 29, 2026 - JavaScript
Switchless RoCE collectives, NCCL profiles and fabric tooling for DGX Spark inference recipes
cuda nvidia rdma roce nccl tensor-parallelism vllm llm-inference distributed-inference gb10 dgx-spark connectx-7
- Updated
Oct 31, 2026 - Python
Mixed-vendor GPU inference cluster manager with speculative decoding
python machine-learning deep-learning metal cuda p2p homelab rocm gpu-cluster llama-cpp gpu-inference local-llm llm-inference ollama speculative-decoding distributed-inference gpu-pooling self-hosted-ai openai-compatible mixed-gpu
- Updated
Jul 2, 2026 - Python