streaming-inference 开源项目推荐（31 个）

paul-krug /pytorch-tcn

（实时）PyTorch 中的时间卷积

ictnlp /SLED-TTS

Streamable Text-to-Speech model using a language modeling approach, without vector quantization

OpenBMB /SimpleMemVLA

Native-video memory for vision-language-action models, using timestamped visual history and exact streaming inference for long-horizon robot manipulation.

Tele-AI /TeleFuser

High-performance runtime for real-time multimodal generation and world models—streaming inference, stateful sessions, and distributed GPU execution.

zhangzijie-pro /Speaker-Verification

Dual-model speech AI toolkit for speaker verification and speaker-aware diarization, with streaming inference, meeting analysis, long-audio monitoring, and speaker-bank integration.

eulogik /NanoForecast

World's most deployable time series foundation model — 200K-6.5M params, zero-shot forecasting, streaming RNN inference, ONNX edge deployment, runs on Raspberry Pi

jiff3 /EchoStream-3D

Quality-aware adaptive streaming segmentation for volumetric ultrasound research

pszemraj /megalodon-hf

Pure PyTorch + 🤗 Transformers reimplementation of Megalodon (CEMA + chunked attention) - readable, hackable, no CUDA kernels required

TheRootKIDD /zaphira

Zaphira is a hybrid resident+streamed inference engine built for one hardware story: running Mistral's large coding models (Devstral 2 123B, Devstral Small 2 24B) on four consumer RTX 3060s.

wpferrell /Bigsmall

Lossless AI model compression - ~34% smaller with bit-identical weights; the autopilot profiles your machine, picks the highest fidelity that runs, and streams models bigger than your RAM.

Brainchip-Inc /tenns-core

Efficient State Space Model layers in pure PyTorch — FFT training, streaming inference, ONNX export for edge deployment

sunny-kavixo /LittleLess-AI

Experimental low-bit weight storage and streaming inference with small-model benchmarks and explicit research limitations.

YazanAi-Dev3 /Vehicle-Fault-Diagnosis-Engine

An end-to-end MLOps pipeline for vehicle diagnostics featuring a temporal LSTM stateful inference engine.

ollycassidy13 /CascadeLUT

CascadeLUT: Information-Ordered Streaming Inference for Bandwidth-Constrained FPGAs [FPL'26]

esl-epfl /streaminnc

Don't Think It Twice: Exploit Shift Invariance for Efficient Online Streaming Inference of CNNs

J-X0 /quenby-metering-streaming-inference-mcp

Streaming inference mcp-server for Quenby Metering — regulated data cannot leave the customer's boundary

SaudMonsour /streamgate-tf

TensorFlow gated linear attention from scratch: byte language modeling, fixed-size streaming state, and measured quality-memory-latency trade-offs.

J-X0 /rooksbridge-labs-streaming-inference-sidecar

Streaming inference daemon-sidecar for Rooksbridge Labs — hard p99 latency budget under 150ms

J-X0 /quarryman-labs-streaming-inference-mcp

Streaming inference mcp-server for Quarryman Labs — every decision must carry a reviewable audit trail

TheRobWoodgate /fabriek

High-performance JAX-to-TensorRT compilation pipeline and decoupled gRPC streaming inference server for quantitative trading architectures.