Video-Depth-Estimation-Rankings-and-Stereo-Video-Conversion-Rankings

Video Depth Estimation Rankings
and Stereo Video Conversion Rankings

Table of Contents

Introduction

• Purpose of this repository
• Awesome Stereo Video Conversion
• Awesome Synthetic RGB-D Image Datasets for Training HD Video Depth Estimation Models
• Awesome Synthetic RGB-D Video Datasets for Training and Testing HD Video Depth Estimation Models

Stereo Video Conversion Rankings

• New models based on MiniMax H3 will deliver a new level of quality and the new rankings will be based on them

Video Depth Estimation Rankings

• 170-frame ScanNet: TAE
• 500-frame Bonn RGB-D Dynamic: δ₁

Single Image Depth Estimation Rankings

• Synth4K: δ₁

Appendices

• Appendix 1: Selection of rankings for this repository (to do)
• Appendix 2: Selection of metrics for the rankings (to do)
• Appendix 3: Rules for qualifying models for the rankings (to do)
• Appendix 4: Notes on the table: Awesome Synthetic RGB-D Video Datasets for Training and Testing HD Video Depth Estimation Models
• Appendix 5: List of all research papers from the above rankings
• Appendix 6: List of all research papers from the column headers of the table: Awesome Synthetic RGB-D Video Datasets for Training and Testing HD Video Depth Estimation Models

---

Purpose of this repository

“The 3D shows you a window into reality; the higher frame rate takes the glass out of the window”
— James Cameron

Researchers, if you have found your way here, please consider developing a new Stereo Video Conversion model based on MiniMax H3.

Awesome Stereo Video Conversion

The following list includes all Stereo Video Conversion methods from the last 19 months, from 1 April 2025 to 12 October 2026. This list was created because there is a significant problem with public access to the latest Stereo Video Conversion models, which makes it difficult for researchers to compare their work with the current state of the art and to use the same test set. Consequently, this also makes it difficult to present a single ranking that showcases all of the best models.

Method	Backbone	Submitted on (arXiv)	Venue	Official   repository	Website code
StereoEngine	Wan2.2-VACE-Fun	8 Oct 2026		-
GRT	ProPainter	6 Jul 2026		-
αDepth	?	29 May 2026		-	-
StereoCrafter2	Wan2.1-VACE-14B	-	-		-
DreamStereo	Wan2.1-1.3B	14 Apr 2026		-
HairGuard	Wan2.1-VACE-1.3B	6 Jan 2026		-	-
StereoPilot	Wan2.1-T2V-1.3B	18 Dec 2025
Elastic3D	SVD	16 Dec 2025		-
StereoWorld	Wan2.1-T2V-1.3B	10 Dec 2025
Restereo	StereoCrafter (based on SVD)	6 Jun 2025		-	-
M2SVid	SVD	22 May 2025
Eye2Eye	Lumiere	30 Apr 2025		-

Awesome Synthetic RGB-D Image Datasets for Training HD Video Depth Estimation Models

Although video depth estimation models should be trained mainly on synthetic RGB-D video datasets I decided to add two synthetic RGB-D image datasets because of their unique features.

	Dataset	Venue	Resolution	Unique features
1	SynthHuman 📌 Human faces 😍		384×512	The dataset contains 98040 samples feature the face, 99976 sample feature the full body and 99992 samples feature the upper body. DAViD trained on this dataset alone achieved better depth estimation results than Depth Anything V2 Large, Depth Pro and even Sapiens-2B on the Goliath-Face test set. See the results in Table 1.
2	MegaSynth		512×512	Huge size: 700K scenes and the incredible improvement in depth estimation results of the fine-tuned Depth Anything V2 ViT-B model on MegaSynth and evaluated on Hypersim. See the results in Table 6.

Awesome Synthetic RGB-D Video Datasets for Training and Testing HD Video Depth Estimation Models

The following list contains only synthetic RGB-D datasets in which at least some of the images can be composited into a video sequence of at least 32 frames. The minimum number of frames was chosen on the basis of the ablation studies shown in Table 5 by the Video Depth Anything researchers.

Most datasets contain ready-to-use video sequences of appropriately numbered images in individual folders, but in the case of the PLT-D3 dataset, images from at least two folders have to be combined to make a longer video sequence and in the case of the ClaraVid dataset, images have to be arranged in the correct order to make a 32-frame video sequence, for example in the order given in Appendix 4.

Researchers, if you are going to use the following list to select datasets to train your models check their quality very carefully and choose the best ones. I have only visually checked a few of them and have marked on the list 2 datasets to check particularly carefully and 2 datasets that in my opinion are not suitable for training video depth estimation models. I have given the reasons for such markings in the same Appendix 4.

In selecting the best datasets, comparisons of their quality can be very helpful, such as in Table 9, Table 6, another Table 6 for depth estimation models and TABLE V plus TABLE IV for stereo matching models, although a similar technique can also be used for depth estimation models.

	Dataset	Venue	Resolution	V G	D A 3 T	M o 3	M o 2	D P	U D 2	G D	V D A	D V D	D C	R D
1	SynthVerse 📌 Human face 4K 😍 in the 164-frame battery-grabbing scene from “Charge”		4096×1716	-	-	-	-	-	-	-	-	-	-	-
2	PhysInOne 📌 Small particles 😍 in the PhysInOneP16		1120×1120	-	-	-	-	-	-	-	-	-	-	-
3	SDG-SynHuman 📌 Human poses 😍 631 million RGB frames!		1920×1080	-	-	-	-	-	-	-	-	-	-	-
4	BEDLAM2.0 📌 Human poses 😍		1280×720	-	-	-	-	-	-	-	-	-	-	-
5	C3I-SynFace 📌 Human faces 😍		640×480	-	-	-	-	-	-	-	-	-	-	-
6	ClaraVid		4032x3024	-	-	-	-	-	-	-	-	-	-	-
7	StereoGenBench		1280×1280	-	-	-	-	-	-	-	-	-	-	-
8	VPM-Data		1280×1280	-	-	-	-	-	-	-	-	-	-	-
9	VP-WorldBench		1280×1280	-	-	-	-	-	-	-	-	-	-	-
10	Spring		1920×1080	T	T	E	E	E	-	-	-	-	-	-
11	SDG-PhyxSim		1920×1080	-	-	-	-	-	-	-	-	-	-	-
12	HorizonGS		1920×1080	-	-	-	-	-	-	-	-	-	-	-
13	PLT-D3		1920×1080	-	-	-	-	-	-	-	-	-	-	-
14	MVS-Synth		1920×1080	T	T	T	T	T	-	T	-	-	-	-
15	SYNTHIA-SF		1920×1080	-	-	-	-	-	-	-	-	-	-	-
16	SynDrone Check before use!		1920×1080	-	-	-	-	-	-	-	-	-	-	-
17	Mid-Air		1024×1024	-	-	T	T	-	-	-	-	-	-	-
18	MatrixCity		1000×1000	T	T	T	T	-	T	-	-	-	T	-
19	StereoCarla		1600×900	-	-	-	-	-	-	-	-	-	-	-
20	LightwheelOcc	-	1600×900	T	-	-	-	-	-	-	-	-	-	-
21	SAIL-VOS 3D		1280×800	-	-	-	-	T	-	-	-	-	-	-
22	SHIFT		1280×800	-	-	-	-	-	-	-	-	-	-	-
23	SYNTHIA-Seqs 🚫 Do not use! 🚫		1280×760	T	-	T	T	-	-	-	-	-	-	-
24	BEDLAM		1280×720	T	-	-	-	T	T	-	-	-	-	-
25	Dynamic Replica		1280×720	T	-	-	-	T	T	T	-	-	T	-
26	OmniWorld-Game		1280×720	T	-	T	-	-	-	-	-	-	-	-
27	WorldRover		1280×720	-	-	-	-	-	-	-	-	-	-	-
28	Syn4D		1280×720	-	-	-	-	-	-	-	-	-	-	-
29	InFlux++ Synth		1280×720	-	-	-	-	-	-	-	-	-	-	-
30	Infinigen SV		1280×720	-	-	-	-	-	-	-	-	-	-	-
31	Infinigen		1280×720	-	-	-	-	-	-	-	-	-	-	-
32	DigiDogs 🚫 Do not use! 🚫		1280×720	-	-	-	-	-	-	-	-	-	-	-
33	Aria Synthetic Environments Check before use!	-	704×704	T	-	-	-	-	-	-	-	-	-	-
34	TartanGround		640×640	T	-	-	-	-	-	-	-	-	-	-
35	TartanAir V2	-	640×640	-	-	-	-	-	-	-	-	-	-	-
36	BlinkVision		960×540	-	-	-	-	-	-	-	-	-	-	-
37	PointOdyssey		960×540	T	T	-	-	-	T	T	T	-	-	E
38	DyDToF		960×540	-	-	-	-	-	-	-	-	-	-	E
39	IRS		960×540	-	T	T	T	T	-	T	T	-	-	-
40	Scene Flow		960×540	-	-	-	-	-	-	-	-	-	-	-
41	THUD++		730×530	-	-	-	-	-	-	-	-	-	-	-
42	TAU Agent		1024×512	-	T	-	-	-	-	-	-	-	-	-
43	TransPhy3D		512×512	T	-	-	-	-	-	-	-	-	-	-
44	3D Ken Burns		512×512	-	T	T	T	T	-	-	-	-	-	-
45	SynPhoRest	-	848×480	-	-	-	-	-	-	-	-	-	-	-
46	TartanAir		640×480	T	T	T	T	T	T	T	T	T	-	T
47	ParallelDomain-4D		640×480	-	-	-	-	-	-	-	-	-	-	-
48	EDEN		640×480	-	T	T	T	T	T	-	-	-	-	-
49	GTA-SfM		640×480	T	T	T	T	-	-	-	-	-	-	-
50	InteriorNet		640×480	-	-	-	-	-	-	-	-	-	-	-
51	SYNTHIA-AL		640×480	-	-	-	-	-	-	-	-	-	-	-
52	MPI Sintel		1024×436	E	E	E	E	E	E	E	E	E	E	-
53	CarlaOcc		1408×376	T	-	-	-	-	-	-	-	-	-	-
54	Virtual KITTI 2		1242×375	-	T	-	-	T	-	T	T	-	-	-
55	Virtual KITTI		1242×375	-	-	-	-	-	-	-	-	T	-	-
56	TartanAir Shibuya		640×360	-	-	-	-	-	-	-	-	-	-	-
	Total: T (training)			15	11	10	9	9	6	6	4	2	2	1
	Total: E (testing)			1	1	2	2	2	1	1	1	1	1	2

170-frame ScanNet: TAE

📝 Note: This ranking is based on the evaluation protocol proposed by Video Depth Anything developers.

RK	Model Links:          Venue   Repository	TAE ↓   {Input fr.} Table 3 FFN	TAE ↓   {Input fr.} Table 2& GD	TAE ↓   {Input fr.} Table 1 VDA
1	FFN	0.380 {MF}	-	-
2	GemDepth-VDA	-	0.47 {MF}	-
3	VDA-L	- VDA-L-Syn: 0.570 {MF}	0.57 {MF} VDA-L-Syn: -	0.570 {MF} VDA-L-Syn: 0.570 {MF}
4	DVD v1.1	-	0.61 {MF}	-
5	DepthCrafter	0.639 {MF}	-	0.639 {MF}
6	RollingDepth	-	0.65 {MF}	-
7	Depth Any Video	0.967 {MF}	-	0.967 {MF}
8	ChronoDepth	1.022 {MF}	-	1.022 {MF}
9	Depth Anything V2 Large	1.140 {1}	1.14 {1}	1.140 {1}
10	NVDS	2.176 {4}	-	2.176 {4}

500-frame Bonn RGB-D Dynamic: δ₁

📝 Note 1: This ranking is based on the evaluation protocol proposed by Video Depth Anything developers.
📝 Note 2: A high rank of the Pixel-Perfect Video Depth model in this ranking does not guarantee that this model is suitable for practical applications, see https://github.com/gangweix/pixel-perfect-depth/issues/27. We are waiting for this model to be fixed and evaluated using temporal stability metrics.

RK	Model Links:          Venue   Repository	δ1 ↑      {Input fr.} Table 3 FFN	δ1 ↑      {Input fr.} TABLE II PPVD	δ1 ↑      {Input fr.} Table 1& GD	δ1 ↑      {Input fr.} Table 1 VDA
1	FFN	0.982 {MF}	-	-	-
2	Pixel-Perfect Video Depth	-	0.979 {MF}	-	-
3	GemDepth-VDA	-	-	0.978 {MF}	-
4	VDA-L	- VDA-L-Syn: 0.961 {MF}	0.959 {MF} VDA-L-Syn: -	0.959 {MF} VDA-L-Syn: -	0.959 {MF} VDA-L-Syn: 0.961 {MF}
5	DVD v1.1	-	-	0.948 {MF}	-
6	RollingDepth	-	0.931 {MF}	0.931 {MF}	-
7	Depth Anything V2 Large	0.864 {1}	0.864 {1}	0.864 {1}	0.864 {1}
8	DepthCrafter	0.803 {MF}	0.803 {MF}	0.803 {MF}	0.803 {MF}
9	NVDS	0.674 {4}	0.674 {4}	0.674 {4}	0.674 {4}
10	ChronoDepth	0.665 {MF}	0.665 {MF}	0.665 {MF}	0.665 {MF}

Synth4K: δ₁

RK	Model Links:          Venue   Repository	δ1 ↑      {Input fr.} Table C.1 MoGe-3
1	MoGe-3 ViT-G Step 3	0.949 {1}
2	MoGe-2	0.913 {1}
3	Depth Anything 3	0.883 {1}
4	InfiniDepth	0.878 {1}
5	UniDepthV2	0.869 {1}
6	Pixel-Perfect Depth	0.868 {1}
7	Depth Pro	0.866 {1}
8	UniK3D	0.862 {1}

Appendix 4: Notes on the table: Awesome Synthetic RGB-D Video Datasets for Training and Testing HD Video Depth Estimation Models

📝 Note 1: Example of arranging images in the correct order to make a 32-frame video sequence for the ClaraVid dataset:

<your-data-path>/008_urban_dense_1/left_rgb/45deg_low_h/00360.jpg
<your-data-path>/008_urban_dense_1/left_rgb/45deg_low_h/00320.jpg
<your-data-path>/008_urban_dense_1/left_rgb/45deg_low_h/00280.jpg
<your-data-path>/008_urban_dense_1/left_rgb/45deg_low_h/00240.jpg
<your-data-path>/008_urban_dense_1/left_rgb/45deg_low_h/00200.jpg
<your-data-path>/008_urban_dense_1/left_rgb/45deg_low_h/00160.jpg
<your-data-path>/008_urban_dense_1/left_rgb/45deg_low_h/00120.jpg
<your-data-path>/008_urban_dense_1/left_rgb/45deg_low_h/00080.jpg
<your-data-path>/008_urban_dense_1/left_rgb/45deg_low_h/00040.jpg
<your-data-path>/008_urban_dense_1/left_rgb/45deg_low_h/00000.jpg
<your-data-path>/008_urban_dense_1/left_rgb/45deg_low_h/00001.jpg
<your-data-path>/008_urban_dense_1/left_rgb/45deg_low_h/00002.jpg
<your-data-path>/008_urban_dense_1/left_rgb/45deg_low_h/00003.jpg
<your-data-path>/008_urban_dense_1/left_rgb/45deg_low_h/00004.jpg
<your-data-path>/008_urban_dense_1/left_rgb/45deg_low_h/00005.jpg
<your-data-path>/008_urban_dense_1/left_rgb/45deg_low_h/00006.jpg
<your-data-path>/008_urban_dense_1/left_rgb/45deg_low_h/00007.jpg
<your-data-path>/008_urban_dense_1/left_rgb/45deg_low_h/00008.jpg
<your-data-path>/008_urban_dense_1/left_rgb/45deg_low_h/00009.jpg
<your-data-path>/008_urban_dense_1/left_rgb/45deg_low_h/00010.jpg
<your-data-path>/008_urban_dense_1/left_rgb/45deg_low_h/00011.jpg
<your-data-path>/008_urban_dense_1/left_rgb/45deg_low_h/00012.jpg
<your-data-path>/008_urban_dense_1/left_rgb/45deg_low_h/00013.jpg
<your-data-path>/008_urban_dense_1/left_rgb/45deg_low_h/00014.jpg
<your-data-path>/008_urban_dense_1/left_rgb/45deg_low_h/00015.jpg
<your-data-path>/008_urban_dense_1/left_rgb/45deg_low_h/00016.jpg
<your-data-path>/008_urban_dense_1/left_rgb/45deg_low_h/00017.jpg
<your-data-path>/008_urban_dense_1/left_rgb/45deg_low_h/00018.jpg
<your-data-path>/008_urban_dense_1/left_rgb/45deg_low_h/00019.jpg
<your-data-path>/008_urban_dense_1/left_rgb/45deg_low_h/00059.jpg
<your-data-path>/008_urban_dense_1/left_rgb/45deg_low_h/00099.jpg
<your-data-path>/008_urban_dense_1/left_rgb/45deg_low_h/00139.jpg

📝 Note 2: Do not use the SYNTHIA-Seqs dataset for training HD video depth estimation models! The depth maps in this dataset do not match the corresponding RGB images. This is particularly evident in the example of tree leaves:
<your-data-path>/SYNTHIA-SEQS-01-SPRING/Depth/Stereo_Left/Omni_F/000071.png
<your-data-path>/SYNTHIA-SEQS-01-SPRING/RGB/Stereo_Left/Omni_F/000071.png.
📝 Note 3: Do not use the DigiDogs dataset for training HD video depth estimation models! The depth maps in this dataset do not match the corresponding RGB images. See the objects behind the campfire, the shifting position of the vegetation on the left and the clear banding on the depth map:
<your-data-path>/DigiDogs2024_full/09_22_2022/00054/images/img_00012.tiff.
📝 Note 4: Check before use the SynDrone dataset for training HD video depth estimation models! The depth maps in this dataset have large white areas of unknown depth, which should not happen with a synthetic dataset. Example depth map:
<your-data-path>/Town01_Opt_120_depth/Town01_Opt_120/ClearNoon/height20m/depth/00031.png.
📝 Note 5: Check before use the Aria Synthetic Environments dataset for training HD video depth estimation models! The depth maps in this dataset have large white areas of unknown depth, which should not happen with a synthetic dataset. Example depth map:
<your-data-path>/75/depth/depth0000109.png.

Appendix 5: List of all research papers from the above rankings

Method	Abbr.	Paper	Venue      (Alt link)	Official   repository
ChronoDepth	-	Learning Temporally Consistent Video Depth from Video Diffusion Priors
Depth Any Video	DAV	Depth Any Video with Scalable Synthetic Data
Depth Anything 3	DA3	Depth Anything 3: Recovering the Visual Space from Any Views
Depth Anything V2	DA V2	Depth Anything V2
Depth Pro	DP	Depth Pro: Sharp Monocular Metric Depth in Less Than a Second
DepthCrafter	DC	DepthCrafter: Generating Consistent Long Depth Sequences for Open-world Videos
DVD	-	DVD: Deterministic Video Depth Estimation with Generative Priors
FFN	-	Forget, Anticipate and Adapt: Test Time Training for Long Videos
GemDepth	GD	GemDepth: Geometry-Embedded Features for 3D-Consistent Video Depth
InfiniDepth	-	InfiniDepth: Arbitrary-Resolution and Fine-Grained Depth Estimation with Neural Implicit Fields
MoGe-2	Mo2	MoGe-2: Accurate Monocular Geometry with Metric Scale and Sharp Details
MoGe-3	Mo3	MoGe-3: Fine-Detail Monocular Geometry Estimation with Self-Guided Sparse Volumetric Refinement
NVDS	-	Neural Video Depth Stabilizer
Pixel-Perfect Depth	PPD	Pixel-Perfect Depth with Semantics-Prompted Diffusion Transformers
Pixel-Perfect Video Depth	PPVD	Pixel-Perfect Visual Geometry Estimation
RollingDepth	RD	Video Depth without Video Models
UniDepthV2	UD2	UniDepthV2: Universal Monocular Metric Depth Estimation Made Simpler
UniK3D	-	UniK3D: Universal Camera Monocular 3D Estimation
Video Depth Anything	VDA	Video Depth Anything: Consistent Depth Estimation for Super-Long Videos

Appendix 6: List of all research papers from the column headers of the table: Awesome Synthetic RGB-D Video Datasets for Training and Testing HD Video Depth Estimation Models

Method	Abbr.	Paper	Venue      (Alt link)	Official   repository
Depth Anything 3	DA3	Depth Anything 3: Recovering the Visual Space from Any Views
Depth Pro	DP	Depth Pro: Sharp Monocular Metric Depth in Less Than a Second
DepthCrafter	DC	DepthCrafter: Generating Consistent Long Depth Sequences for Open-world Videos
DVD	-	DVD: Deterministic Video Depth Estimation with Generative Priors
GemDepth	GD	GemDepth: Geometry-Embedded Features for 3D-Consistent Video Depth
MoGe-2	Mo2	MoGe-2: Accurate Monocular Geometry with Metric Scale and Sharp Details
MoGe-3	Mo3	MoGe-3: Fine-Detail Monocular Geometry Estimation with Self-Guided Sparse Volumetric Refinement
RollingDepth	RD	Video Depth without Video Models
UniDepthV2	UD2	UniDepthV2: Universal Monocular Metric Depth Estimation Made Simpler
Video Depth Anything	VDA	Video Depth Anything: Consistent Depth Estimation for Super-Long Videos
ViGeo	VG	Towards Consistent Video Geometry Estimation

List of research papers to be added to the rankings

Method	Abbr.	Paper	Venue      (Alt link)	Official   repository
StereoEngine	-	StereoEngine: Scaling Stereo Bimanual Robot Data without Stereo Cameras		-
GRT	-	Geometric Reciprocity: Unlocking Self-Supervision for Stereoscopic Video Generation		-
αDepth	-	αDepth: Learning Single-Pass Soft Boundary Decomposition for Stereo Conversion		-
DreamStereo	-	DreamStereo: Towards Real-Time Stereo Inpainting for HD Videos		-
HairGuard	-	Guardians of the Hair: Rescuing Soft Boundaries in Depth, Stereo, and Novel Views		-
StereoPilot	-	StereoPilot: Learning Unified and Efficient Stereo Conversion via Generative Priors
Elastic3D	-	Elastic3D: Controllable Stereo Video Conversion with Guided Latent Decoding		-
StereoWorld	-	StereoWorld: Geometry-Aware Monocular-to-Stereo Video Generation
Restereo	-	Restereo: Unifying diffusion stereo video generation and restoration		-
M2SVid	-	M2SVid: End-to-End Inpainting and Refinement for Monocular-to-Stereo Video Conversion
Eye2Eye	-	Eye2Eye: A Simple Approach for Monocular-to-Stereo Video Synthesis		-
StereoCrafter	-	StereoCrafter: Diffusion-based Generation of Long and High-fidelity Stereoscopic 3D from Monocular Videos
SVG	-	SVG: 3D Stereoscopic Video Generation via Denoising Frame Matrix