TensorFusion.AI Less GPUs, More AI Apps. Explore the docs » View Demo Report Bug Request Feature Tensor Fusion is a state-of-the-art GPU virtualization and pooling solution designed to optimize GPU cluster utilization to its fullest potential. 🌟 Highlights 📐 Fractional Virtual GPU 🔄 Remote GPU Sharing over Ethernet/InfiniBand ⚖️ GPU-first Scheduling and Auto-scaling 📊 GPU Oversubscription and VRAM Expansion 🛫…
This repository builds the two components that run in your cluster:
Controller: the CRD controllers, the Pod mutating webhook, and a GPU-aware scheduler built on kube-scheduler, in one Deployment. It also drives node scaling, directly through cloud APIs or through Karpenter.
Hypervisor: a DaemonSet Pod on every GPU node, written in Go. It discovers devices through vendor accelerator libraries, reports them as GPU and GPUNode resources, and enforces each workload’s compute and VRAM limits.
A workload asks for a fraction of a GPU with Pod annotations. The webhook injects what the chosen isolation mode needs (shared, soft, hard, or partitioned), the scheduler picks the GPU, and the hypervisor enforces the limits. Metrics are written to a GreptimeDB instance inside the cluster.
Remote GPU sharing with SOTA GPU-over-IP technology, less than 4% performance loss
GPU VRAM expansion and hot/cold tiering
Non-NVIDIA GPU/NPU vendor support
Pooling & Scheduling & Management
GPU/NPU pool management in Kubernetes
GPU-first scheduling and allocation, with 1 TFLOPs, 1% Computing, 1 MB precision
GPU node auto provisioning/termination, Karpenter integration
GPU compaction/bin-packing
Take full control of GPU allocation with precision targeting by vendor, model, device index, and more
Seamless onboarding experience for PyTorch, TensorFlow, llama.cpp, vLLM, TensorRT, SGLang and all popular AI training/serving frameworks
Seamless migration from existing NVIDIA operator and device-plugin stack
Centralized Dashboard & Control Plane
GPU-first autoscaling policies, auto set requests/limits/replicas
Request multiple vGPUs with group scheduling for large models
Support different QoS levels
Hardware partitioned mode isolation like NVIDIA Dynamic MIG
Support Kubernetes dynamic resource allocation (DRA) API
Enterprise Features
GPU live-migration, snapshot and restore GPU context cross cluster
AI model registry and preloading, build your own private MaaS(Model-as-a-Service)
Advanced auto-scaling policies, scale to zero, rebalance of hot GPUs
Advanced observability features, detailed metrics & tracing/profiling of CUDA calls
Monetize your GPU cluster by multi-tenancy usage measurement & billing report
Enterprise level high availability and resilience, support topology aware scheduling, GPU node auto failover etc.
Enterprise level security, complete on-premise deployment support
Enterprise level compliance, SSO/SAML support, advanced audit, ReBAC control, SOC2 and other compliance reports available
🗳️ Platform Support
Run on Linux Kubernetes clusters
Run on Linux VMs or Bare Metal (one-click onboarding to Edge K3S)
Run on Windows (Not open sourced, contact us for support)
See the open issues for a full list of proposed features (and known issues).
🙏 Contributing
Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.
If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also simply open an issue with the tag “enhancement”.
Don’t forget to give the project a star! Thanks again!
Development Setup
This project patches Kubernetes scheduler components. Before building or running tests, you need to vendor dependencies and apply patches:
make vendor # Vendor dependencies and apply scheduler patches
make build # Build the controller
make build-hypervisor # Build the hypervisor
make test # Run the test suite (envtest)
make ut F=<file> # Run one controller test file
End-to-end tests against real clusters are in e2e/: Amazon EKS with Karpenter, and Alibaba Cloud ACK with Karpenter on real GPU nodes.
How to Contribute
Fork the Project
Create your Feature Branch (git checkout -b feature/AmazingFeature)
Make your changes and run linter (make lint-fix)
Run tests to ensure everything works (make test)
Commit your Changes (git commit -m 'Add some AmazingFeature')
Push to the Branch (git push origin feature/AmazingFeature)
Open a Pull Request
Top contributors
🔷 License
TensorFusion main repo is open sourced with Apache 2.0 License, which includes GPU pooling, scheduling, management features, you can use it for free and customize it as you want.
The vGPU hypervisor is part of this repository since v2 (cmd/hypervisor, pkg/hypervisor) and is covered by the same license. The vgpu.rs repo, which held the original Rust hypervisor and Fractional GPU limiter, is open sourced with the Apache 2.0 License as well.
Advanced GPU virtualization and GPU-over-IP sharing features are also free to use when GPU total number of your organization is less than 10, but the implementation is not fully open sourced, please contact us for more details.
Features mentioned in “Enterprise Features” above are paid, licensed users can use these features in TensorFusion Console.
For large scale deployment that involves non-free features of #3 and #4, please contact us, pricing details are available here
![dependabot[bot]](https://avatars.githubusercontent.com/in/29110?v=4)
评论