UASOL, a large-scale high-resolution outdoor stereo dataset

Zuria Bauer, Francisco Gomez‐Donoso, Edmanuel Cruz, Sergio Orts‐Escolano, Miguel CazorlaScientific 2019 · DOI

TL;DR

This paper introduces UASOL, a large-scale, high-resolution outdoor stereo dataset captured from a pedestrian's egocentric perspective. It addresses the need for more realistic and diverse ground-truth data for training deep learning models for depth estimation, which are currently limited by the availability of such datasets.

What Problem Does This Paper Address?

Accurate depth estimation is crucial for many computer vision tasks, including autonomous driving, robotics, and augmented reality. While deep learning models have shown great promise in this area, they require vast amounts of high-quality, ground-truth data for training. Existing datasets often fall short in several key aspects: many are synthetic and lack realism, others are captured from a vehicle's perspective (not egocentric), and few offer the high resolution and diversity needed to generalize well to real-world scenarios. This gap hinders the development of robust depth estimation systems for applications that require a pedestrian's viewpoint.

Methods

The UASOL dataset was captured using a ZED stereo camera, which provides synchronized RGB stereo pairs and depth maps. The camera was mounted on a strap worn by a pedestrian, ensuring an egocentric viewpoint. To enhance data quality and realism, the ZED camera was set to its maximum resolution (2208x1242 pixels) at 15 fps, and a GlobalSat GPS device was used to record location data. The dataset includes depth maps generated by two methods: the camera's built-in Semi-Global Matching (SGBM) algorithm and a novel deep-learning-based stereo method called GC-Net. For validation, the authors employed YOLOv3 for object counting and a segmentation method for pixel-wise classification, allowing for quantitative analysis of the dataset's content and diversity.

Datasets

The primary contribution of this paper is the introduction of the UASOL dataset. This dataset comprises 482,736 frames across 33 sequences, captured from a pedestrian's perspective in various outdoor environments. It offers high-resolution RGB images and corresponding depth maps, along with GPS data and positional tracking. The authors also reference and compare their work against existing datasets such as SYNTHIA, KITTI, ETH3D, Tanks and Temples, and Middlebury, highlighting the limitations of these datasets in terms of realism, viewpoint, and scale for outdoor egocentric depth estimation.

Key Insights and Ideas

  • Egocentric Outdoor Depth is Crucial: The paper strongly emphasizes that for applications involving pedestrian navigation or interaction, egocentric outdoor depth data is paramount, and existing datasets are insufficient in this regard.
  • Realism and Diversity Challenge Deep Learning: The UASOL dataset's focus on real-world variability (different times of day, weather conditions, human interaction) is designed to push the boundaries of deep learning model generalization.
  • Dual Depth Map Approach: Providing depth maps from both a traditional stereo algorithm (SGBM) and a deep learning method (GC-Net) offers a valuable benchmark for researchers to compare their own models against established baselines without needing to retrain them.
  • High Resolution Matters: The choice of high resolution (2208x1242) is intended to capture finer details crucial for accurate depth estimation, especially in complex outdoor scenes.

Key Results

  • The UASOL dataset contains approximately 321,804 RGB images and an equal number of depth maps.
  • The ZED camera's SGBM algorithm provides depth maps with a mean error of 493.93 mm, while the GC-Net algorithm achieves a mean error of 180.52 mm on synthetic data.
  • The paper demonstrates that training a monocular depth prediction network on the UASOL dataset significantly improves its performance compared to using a publicly available model trained on other datasets, particularly in handling challenging natural conditions like shadows and direct sunlight.
  • The dataset includes detailed object counts and pixel-wise class distributions across sequences, providing insights into the semantic content and variability.

Limitations and Future Work

  • SGBM vs. GC-Net Performance: While GC-Net generally outperforms SGBM, the paper notes that SGBM performs better in low-texture areas and at greater distances (around 8 meters), whereas GC-Net is more precise at closer distances. This highlights the trade-offs between different depth estimation approaches.
  • GPS Accuracy: The GPS device has an error of less than 3 meters, which might be a limitation for applications requiring highly precise localization.
  • Future Work: The authors plan to focus on applying SLAM techniques to the dataset for complete 3D reconstruction of the university campus, enabling the creation of a virtual tour. They also aim to release Python scripts for data reading, GPS processing, and manifest file handling to facilitate dataset usage.

Summary updated

Abstract

In this paper, we propose a new dataset for outdoor depth estimation from single and stereo RGB images. The dataset was acquired from the point of view of a pedestrian. Currently, the most novel approaches take advantage of deep learning-based techniques, which have proven to outperform traditional state-of-the-art computer vision methods. Nonetheless, these methods require large amounts of reliable ground-truth data. Despite there already existing several datasets that could be used for depth estimation, almost none of them are outdoor-oriented from an egocentric point of view. Our dataset introduces a large number of high-definition pairs of color frames and corresponding depth maps from a human perspective. In addition, the proposed dataset also features human interaction and great variability of data, as shown in this work.
BibTeX
@inproceedings{bauer2019uasol,
  author = {Zuria Bauer and Francisco Gomez‐Donoso and Edmanuel Cruz and Sergio Orts‐Escolano and Miguel Cazorla},
  title = {{UASOL, a large-scale high-resolution outdoor stereo dataset}},
  booktitle = {Scientific},
  year = {2019},
  doi = {10.1038/s41597-019-0168-5},
}
Figure