Abstract
Monocular depth estimation models are frequently limited by the specific perspectives of the datasets on which they are trained, such as autonomous driving (KITTI) or indoor environments (NYUv2). This paper presents the Rigid Depth Constructor (RDC), an open-source framework designed to democratize the creation of high-quality depth validation datasets. By decoupling Lidar-based environment scanning from camera-based video acquisition, RDC enables the benchmarking of algorithms in diverse, real-world scenarios—including Unmanned Aerial Vehicle (UAV) flight—without the need for expensive, heavy, and perfectly calibrated rigid rigs.
Introduction: The Benchmark Bottleneck
In computer vision research, models are often "benchmarked into a corner." While datasets like KITTI and NYUv2 are foundational, they represent narrow environmental slices. A model optimized for highway navigation may fail when applied to a drone navigating a forest or a robot in a warehouse.
The primary barrier to expanding these benchmarks is the difficulty of generating "ground truth" depth data. Traditionally, this requires a "rigid rig"—a heavy setup where high-end Lidar and cameras are physically coupled and calibrated. Such hardware is often inaccessible to small teams and impractical for lightweight platforms like consumer drones.
As shown in the table above, existing benchmarks often lack the camera orientation and position variation necessary for robust UAV applications. RDC addresses this by removing the requirement for simultaneous, rigid hardware integration.
Methodology: The RDC Pipeline
RDC utilizes a "scan-then-localize" workflow that provides maximum flexibility for researchers. The process begins with an environment scan using any Lidar sensor to create a 3D point cloud. Subsequently, standard video is recorded in the same environment.
The RDC pipeline processes this data through several key stages:
- Photogrammetry: COLMAP (Structure from Motion) is used on a subset of video frames to build a sparse 3D model.
- Registration: This sparse model is aligned to the Lidar "ground truth" using Iterative Closest Point (ICP) algorithms, localizing the camera within the scan.
- Linear-Time Localization: To maintain computational efficiency, RDC localizes the remaining video frames against the established model in linear time.
- Filtering and Projection: A Savitzky-Golay filter smooths the camera trajectory, and Lidar points are projected onto camera frames to create dense depth maps.
New Benchmarks: Manoir and University Hall
To demonstrate the tool's utility, the authors introduced two new datasets:
- Manoir Dataset: High-resolution (4K/30fps and 720p/120fps) UAV footage of a mansion, featuring complex geometries like sloped roofs and dense foliage.
- University Hall Dataset: An indoor scene of a large hall with high ceilings, captured via handheld Lidar and drone-mounted cameras.
These datasets highlight the "context is king" principle. When testing established models like DepthNet and SFMLearner, performance varied significantly between car-level and UAV-level perspectives. For instance, in the Manoir dataset, DepthNet outperformed SFMLearner (MAE of 11.99m vs 18.40m), though both struggled with very low depth values.
Discussion and Limitations
A key insight from this work is the power of hardware decoupling. By mapping an area once with high-end Lidar, researchers can subsequently use any lightweight camera—including smartphones—to generate valid data. Furthermore, the authors observed that vision-based localization (via COLMAP) can sometimes prove less noisy than traditional Lidar/IMU odometry.
However, RDC is currently limited to static scenes; dynamic elements like moving vehicles or swaying trees can corrupt depth maps. Additionally, while the hardware requirements are low, the computational cost of running SfM on thousands of frames remains high, and the ICP registration step still requires human verification.
Conclusion
The Rigid Depth Constructor offers a path forward for researchers looking to validate depth estimation models beyond standard benchmarks. By lowering the hardware barrier to ground truth generation, RDC facilitates the development of more robust, context-aware computer vision systems for diverse robotics and UAV applications.
Summary updated