forge

███████╗ ██████╗ ██████╗  ██████╗ ███████╗
██╔════╝██╔═══██╗██╔══██╗██╔════╝ ██╔════╝
█████╗  ██║   ██║██████╔╝██║  ███╗█████╗
██╔══╝  ██║   ██║██╔══██╗██║   ██║██╔══╝
██║     ╚██████╔╝██║  ██║╚██████╔╝███████╗
╚═╝      ╚═════╝ ╚═╝  ╚═╝ ╚═════╝ ╚══════╝

⚒ Robotics Data Toolkit & Data Engine ⚒

Forge is two things that share one core:

• A format toolkit — convert, inspect, score, lint, filter, segment, and visualize a single dataset across RLDS, LeRobot, HDF5, MCAP, Zarr, Rosbag, and more.
• A data engine — register every episode you collect into an append-only catalog, then query it with SQL, search it by natural language, dedup and curate it, and explore it in a visual Studio.

Everything works on local paths, hf:// datasets, and s3:// / gs:// buckets.

Install

pip install forge-robotics                  # base CLI + LeRobot v3 read/write
pip install "forge-robotics[all]"           # everything

Pick only the extras you need:

Heads up: the published PyPI build is the stable format toolkit. The data-engine features — catalog, semantic search, dedup/curation, and Forge Studio — are newer than the last release tag. To use those today, install from source:

git clone https://github.com/arpitg1304/forge.git && cd forge
pip install -e ".[all]"

git clone https://github.com/BerkeleyAutomation/robodm.git
pip install -e robodm

Try it in 60 seconds

— pick a public LeRobot dataset, score every episode on 8 quality metrics, drill into the worst demos. No GPU, no auth, ~60 seconds.

Or locally:

forge inspect hf://lerobot/pusht                          # what's in it?
forge convert hf://lerobot/pusht ./out --format lerobot-v3  # convert it
forge quality hf://lerobot/pusht                          # score every episode

Forge speaks a hub-and-spoke architecture: any reader → Episode/Frame → any writer. Add a reader, get every writer for free — no N×M conversion logic. See docs/architecture.md.

The format toolkit

Per-dataset operations. Every command takes a local path, a registry id (droid), an hf:// URL, or an s3:// / gs:// URI.

Convert & interop

forge inspect ./dataset                                    # structure, schema, cameras
forge convert ./rlds_dataset ./out --format lerobot-v3     # convert between formats
forge convert ./data.zarr ./out --format lerobot-v3 --visualize

For complex conversions, generate a YAML config: forge inspect ds/ --generate-config config.yaml, then forge convert ds/ out/ --config config.yaml. See docs/configuration.md.

Score & clean

Quality — inspect motion, timing, and optional video checks. The 0–10 score is a heuristic; unavailable scores remain null.

forge quality ./my_dataset --visualize --export quality.json
forge quality ./my_dataset --video --video-level motion --visualize

• Detect held commands, measured-motion pauses, direction reversals, and timestamp gaps.
• Inspect smoothness, action entropy, and path length. Configure gripper channels and physical action limits explicitly.
• Enable tracking-stall candidates with known absolute-position action/state mappings.
• Add camera coverage, decode failures, blur, exposure, freeze candidates, and estimated observation gaps with --video. Requires [video].
• Review evidence on camera timelines and action/state plots. A flag is a review candidate, not a task-failure label.

Details: quality · video quality · viewer · validation and benchmarks.

Lint — check dataset hygiene (missing task strings, ambiguous cameras, low-res / single-view, missing action fields) against HuggingFace’s LeRobot guidelines. Exits non-zero, so it drops into CI.

forge lint ./my_dataset --strict          # fail on warnings too

Filter — drop bad episodes by quality, flags, or ids → a new dataset.

forge filter ./my_dataset ./clean --min-quality 6.0 --exclude-flags jerky,mostly_static

Dedup — remove near-duplicate episodes within one dataset by perceptual hashing of keyframes (numpy only, no model).

forge dedup ./my_dataset ./deduped --threshold 0.05

Details: lint · filter · dedup.

Understand

Segment — split episodes into phases (reach / grasp / place) via PELT changepoint detection on proprio.

forge segment hf://lerobot/droid_100 --export segments.json --plot timeline.png

Tokenize — turn continuous actions into discrete tokens for VLA training; benchmark strategies on your data.

forge tokenize compare ./my_dataset --sample 20            # recon error / vocab util
forge tokenize write ./my_dataset ./tokenized --strategy openvla-bins

Built-ins: uniform-bins (RT-1), openvla-bins, quantile-bins, mu-law.

Visualize — three backends: browser (default), matplotlib, and Rerun (cameras + time-series on one timeline).

forge visualize pusht                       # web (no install)
forge visualize pusht --backend rerun --segment

Details: segment · tokenize.

The data engine

The toolkit works one dataset at a time. The catalog turns Forge into a system of record: an append-only set of Parquet tables that registers every episode you ingest and annotates it with quality, embeddings, and curation decisions — all queryable with SQL. It’s zero-server (Parquet + embedded DuckDB), lives on a local dir or an s3:// / gs:// bucket, and is readable by pandas / Polars / Spark without Forge.

ingest ──▶ query ──▶ embed ──▶ search ──▶ dedup ──▶ curate ──▶ Studio
                                                              (snapshot → soon)

pip install "forge-robotics[catalog]"       # + [embed] for search, [video] for Studio thumbnails

1. Ingest & query

forge catalog init ./forge-catalog                         # local dir or cloud bucket
forge ingest ./my_dataset -c ./forge-catalog               # register + quality-score each episode
forge ingest s3://lab-bucket/raw/2026-07-18/ -c ./forge-catalog   # re-runs are a no-op (content hash)

forge query "SELECT task, count(*) FROM episodes GROUP BY task" -c ./forge-catalog
forge catalog stats -c ./forge-catalog

Ingestion reuses the same readers as forge inspect and the same scorer as forge quality, so the catalog stays consistent with the toolkit. Writes go through pyarrow; reads through DuckDB (views: episodes, quality_scores, v_latest_quality).

from forge.catalog import Catalog
from forge.catalog.ingest import ingest

cat = Catalog.init("s3://lab-bucket/forge-catalog")
ingest(["s3://lab-bucket/raw/2026-07-18/"], cat)
df = cat.sql("SELECT robot, avg(overall_score) FROM episodes "
             "JOIN v_latest_quality USING(episode_id) GROUP BY robot").to_pandas()

2. Search semantically

Embed episodes with SigLIP (a shared image–text model), then search by natural language — text queries match episode video, not just metadata.

pip install "forge-robotics[embed]"

forge embed -c ./forge-catalog                             # GPU auto: CUDA → Apple MPS → CPU
forge search "picks up the red cup" -c ./forge-catalog --top 10
forge search --like <episode_id> -c ./forge-catalog        # visually-similar episodes

Vectors are versioned per model (siglip-so400m@<ckpt-hash>) and stored in the catalog. Details: forge/embed/README.md.

3. Dedup & curate

Find near-duplicate episodes across the whole corpus (cosine over embeddings), then curate a clean, labeled training set. Near-dup pairs are recorded as facts; which episode wins is decided by policy at curation time.

forge catalog dedup -c ./forge-catalog --threshold 0.97    # store near-dup pairs (dedup_edges)

forge curate -c ./forge-catalog \
    --where "overall_score > 6 AND task = 'pick_place'" \
    --dedup 0.97 --dedup-policy keep-higher-quality --label approved

Curation isn’t only SQL-driven — semantic search and visual review can drive it too. Save a search result (or Studio’s keep/reject decisions) to a selection file and label it, provenance included:

forge search "regrasp after a failed pick" -c ./forge-catalog --top 20 --save sel.json
forge curate -c ./forge-catalog --from sel.json --label approved     # or --ids e1,e2,e3

Curation is an append-log (curation_labels, latest-wins) — nothing is deleted. Policies: keep-higher-quality, keep-longer, keep-first.

4. Forge Studio

A self-contained, themed HTML app to explore the catalog — Overview · Corpus (thumbnails + quality rings) · Dedup review (keep/reject pairs) · Snapshot. One shareable file, no server; real data and video thumbnails embedded.

forge studio -c ./forge-catalog -o studio.html && open studio.html

Details, storage layout, and commit protocol: forge/catalog/README.md. A ready-to-explore example catalog (droid_100) ships in the repo.

Reference

Cloud storage (S3 & GCS)

Every command that takes a dataset or catalog path also accepts s3:// and gs:// URIs. Cloud datasets are downloaded to a temp dir on first access and cleaned up automatically, so every format works exactly as it does locally.

pip install "forge-robotics[s3]"            # or [gcs]
forge inspect s3://my-bucket/datasets/run_0413
forge convert gs://lab-data/rosbags ./out --format lerobot-v3

Streaming reads. For LeRobot-v3 and Zarr, forge inspect and forge ingest read over the network with range requests instead of downloading. Inspecting a 464 MB cloud LeRobot dataset fetches ~3 KB (its info.json); ingesting it — register + quality-score every episode — reads only the proprio parquet columns (~3 MB), never the videos. forge inspect --deep forces a full download when you need per-episode stats; embeddings/video quality still fetch frames.

Auth uses each provider’s standard credential chain (AWS env vars / profiles / IAM roles; GCP Application Default Credentials) — Forge never handles credentials itself. Catalogs can be read and written in the cloud; per-format conversion outputs are local-only for now. Full guide + connectivity troubleshooting: forge/io/README.md.

Dataset registry

A curated catalog of 23+ prominent robotics datasets — browse and use by name instead of memorizing URIs. Browse online

forge registry list --format rlds --embodiment franka      # filter
forge registry search "franka manipulation"
forge inspect droid                                        # ids work in any command
forge demo                                                 # download + inspect + score a demo

Add datasets via forge/registry/CONTRIBUTING.md.

Supported formats

Which models use which format: docs/model_formats.md · format specs: docs/format_reference.md.

Command cheatsheet

Toolkit (per dataset): inspect · convert · quality · lint · filter · dedup · segment · tokenize · visualize · stats · export-video

Data engine (per catalog): catalog init · ingest · query · catalog stats · embed · search · catalog dedup · curate · studio

Discovery: hub · local · registry · demo · formats · version

Full reference: docs/cli.md. Run forge <command> --help for any command.

Roadmap

• Dataset registry — curated catalog of 23+ datasets with CLI + HTML browser
• MCAP first-class support — read + write, ROS2 CDR + Foxglove, no ROS install
• Episode filtering, quality scoring, linting, segmentation, tokenization
• Cloud storage — s3:// / gs:// on every command
• The catalog — ingest, SQL query, semantic search, dedup, curation, Studio
• Snapshots + export — freeze a curated selection → LeRobot/RLDS for training
• [~] Streaming reads — forge inspect + forge ingest stream LeRobot-v3 / Zarr (metadata + proprio via range reads, no video download); frame/embedding streaming next
• Dataset merging & splitting — combine datasets; stratified train/val/test
• Depth / point-cloud support · GR00T writer · distributed conversion

Development

make venv && source .venv/bin/activate
make install-dev
make test

Contributions welcome. See docs/architecture.md for the design.

License

MIT