Splatline

Splatline

Convert 2D videos and photos into interactive 3D scenes using pluggable Gaussian-splatting backends, Rerun, and the SuperSplat editor. MIT-licensed. Runs on macOS — no CUDA required.

v2.0.0 adds three new reconstruction backends (VGGT, DepthSplat, LongSplat) alongside SHARP and TripoSplat, a tiered human motion pipeline, an Electron desktop player with true Gaussian splat rendering, and a FastAPI + SSE backend. See CHANGELOG.md.

Demo

Download the full demo video | Thumbnail

Quick Start

# Install
pip install -r requirements.txt
pip install sharp
npm install  # for the Electron player

# Convert a video to 3D Gaussian splats
python run_video_3d.py

# View in Rerun (point cloud or solid mesh)
python run_video_3d.py          # point cloud mode
python run_video_3d.py solid    # solid mesh mode

# Or view in the Electron desktop player (true splat rendering)
python run_video_electron.py --output-dir output_grok_3d

# Or in the browser-based SuperSplat editor
python run_video_splat.py --output-dir output_grok_3d

The SHARP model (~2.5GB) downloads automatically on first run.

3D viewer controls: Left-drag to rotate, right-drag to pan, scroll to zoom, double-click to reset.

Reconstruction Backends

Five backends, swappable with --splat-backend <name>:

python run_video_3d.py --splat-backend longsplat

See docs/SPLAT_MODELS.md for details.

What v2 fixes

Human Pipeline

Tiered human reconstruction with --human-tier skeleton|mesh|both:

• Skeleton: MotionBERT (ICCV 2023) — temporal 3D pose lifting across 243 frames, fused with splat depth for metric scale
• Mesh: HMR 2.0 / 4DHumans — SMPL body mesh recovery + PHALP identity tracking
• 2D detection: YOLO26-pose (default) or RTMPose (Apache-2.0)

See docs/ATHLETE_TWIN.md.

Viewers

Electron Desktop Player (run_video_electron.py)

Native desktop app with true Gaussian splat rendering via the SuperSplat (PlayCanvas) engine:

python run_video_electron.py --output-dir output_grok_3d

• GPU-accelerated splat compositing (not point clouds)
• 2D source video synced side-by-side at native 24fps
• Smooth timeline playback with on-demand frame loading and preloading
• Flash-free frame swapping (overlapping entity destruction)
• Voxel-grid + opacity subsampling to 600K splats for smooth real-time playback

Requirements: Node.js, npm install, pre-existing 3D output.

SuperSplat Web Viewer (run_video_splat.py)

Same rendering engine, runs in a browser:

python run_video_splat.py --output-dir output_grok_3d

Rerun Viewers

# Image to 3D mesh
python run_image_3d.py photo.jpg --res 1024

# SLAM mapping
python run_slam_3d.py video.mp4

# View a PLY file
python run_splat_viewer.py output_grok_3d/gaussians/frame_000000.ply

FastAPI Backend

python ui/server.py

FastAPI + SSE backend with real-time progress streaming, Pydantic validation, and OpenAPI docs at /docs. Endpoints: /api/config, /api/backends, /api/tiers, /api/jobs/{id}/stream (SSE).

Project Structure

splatline/
├── run_video_3d.py             # Video → 3D (point cloud or solid mesh)
├── run_image_3d.py             # Image → high-res solid 3D mesh
├── run_slam_3d.py              # Monocular SLAM 3D mapping
├── run_splat_viewer.py         # View any PLY as splats in Rerun
├── run_video_splat.py          # SuperSplat editor video viewer (web)
├── run_video_electron.py       # Electron desktop video player
├── ui/server.py                # FastAPI + SSE backend
├── electron/                   # Electron app (main, video-main, preload)
├── src/                        # React + Vite frontend
├── utils/
│   ├── backends/               # VGGT, DepthSplat, LongSplat backends
│   ├── human/                  # MotionBERT, HMR 2.0, PHALP tracking
│   ├── splat_models.py         # Backend registry
│   └── ...                     # Navigation, pathfinding, visualization
├── scripts/                    # v1 converters, visualizers, creative tools
├── docs/                       # SPLAT_MODELS.md, ATHLETE_TWIN.md
├── package.json                # Node.js / Electron config
└── requirements.txt            # Python dependencies

Setup

Python

pip install -r requirements.txt
pip install sharp  # Apple's 3DGS model (auto-downloads ~2.5GB on first run)

Or install SHARP from source: apple/ml-sharp

Electron (optional, for desktop player)

npm install

Verify

python -c "import rerun; import numpy; import torch; import sharp; print('OK')"

System Requirements

• Python 3.8+, Node.js (for Electron)
• macOS, Linux, or Windows
• 8GB RAM minimum (16GB recommended)
• GPU optional: CUDA, MPS (Apple Silicon), or CPU

Troubleshooting

ModuleNotFoundError: No module named 'sharp' → pip install sharp or install from source

Rerun viewer is blank / version mismatch → pip install rerun-sdk==0.23.1 to match your viewer, or update the viewer to match the SDK.

Out of memory during processing → Use --max-frames 10 or lower resolution videos.

GPU not detected → CUDA: pip install torch --index-url https://download.pytorch.org/whl/cu118 → Apple Silicon: MPS is automatic. Use mps device. → CPU: python run_video_3d.py --device cpu (slower but works)

Credits

• Rerun — visualization SDK (Apache-2.0)
• Apple ML-SHARP — monocular 3DGS model
• SuperSplat — Gaussian splat editor (PlayCanvas engine)
• VGGT — geometry foundation model
• DepthSplat — multi-view depth-conditioned splatting
• LongSplat — video-native coherent 3DGS
• MotionBERT — temporal 3D pose lifting
• HMR 2.0 — SMPL body mesh recovery

See NOTICE.md for the full third-party license breakdown.

License

Splatline is MIT-licensed. Some backends have different licenses:

• SHARP — non-commercial research only. Use TripoSplat (MIT) for commercial work.
• VGGT checkpoint — CC-BY-NC (code is MIT)
• LongSplat — check NVlabs terms before commercial use
• YOLO26-pose — AGPL-3.0 (local desktop use is fine; networked services need a commercial license)

See NOTICE.md for details.