EU Long-term Dataset with Multiple Sensors for Autonomous Driving

Zhi Yan, Li Sun, Tomás Krajník, Yassine Ruichek

TL;DR

This paper introduces a robust multisensor platform and a comprehensive new dataset specifically designed for long-term autonomous driving research. By capturing the same urban routes across different seasons, weather conditions, and times of day, the authors provide a vital resource for developing "self-aware" localization and mapping algorithms that can survive real-world environmental shifts.

What Problem Does This Paper Address?

Most autonomous driving (AD) datasets are "snapshots" in time—they capture a few hours of driving in fair weather. However, for a vehicle to be truly autonomous, it must achieve long-term autonomy. This requires navigating the same streets whether it’s a sunny summer afternoon or a snowy winter night.

The authors identify a gap in existing data: a lack of high-frequency, multisensor recordings that capture periodic environmental changes (seasonal/daily) and complex traffic scenarios like roundabouts. They also address the engineering challenge of building a platform that synchronizes eleven heterogeneous sensors to provide a seamless 360-degree view of the environment.

Methods

The authors developed an instrumented vehicle using a ROS-based (Robot Operating System) software architecture. Key technical highlights include:

  • Sensor Fusion & Placement: The platform integrates 11 sensors, including two Velodyne 32-layer lidars, two stereo cameras, two fisheye cameras, a 2D lidar for road markings, radar, IMU, and GNSS-RTK.
  • Interference Mitigation: To use two Velodyne lidars side-by-side without them "blinding" each other, the team utilized a hardware phase-locking feature. This synchronized the laser firings so they wouldn't overlap in the same spatial region at the same time.
  • Calibration: Cameras were calibrated using standard chessboard patterns, while lidar-to-camera extrinsic parameters were estimated by minimizing voxel-wise $L_2$ distances between point clouds and image features.
  • Privacy Compliance: In accordance with EU GDPR, the authors used deep learning-based methods to post-process images, blurring faces and license plates before public release.

Datasets

The EU Long-term Dataset (UTBM Robocar Dataset) is the primary contribution. It features:

  • Scale: 63.4 km of total driving data recorded in Montbéliard, France.
  • Temporal Diversity: 11 rounds of a specific 5km loop recorded at different times (day/night) and seasons (summer/winter/snow).
  • Complex Maneuvers: A dedicated subset focusing on roundabouts (10 different ones), which are notoriously difficult for AD systems to predict.
  • Format: Provided in rosbag format for easy integration with existing robotics pipelines.

Key Insights and Ideas

  • The "See-Think-Act" Cycle: The paper frames AD as a continuous loop where perception and localization are the most critical "essential parts" for safety.
  • Sensor Complementarity: The authors argue against relying on a single sensor type. For example, while 3D lidar is accurate, it struggles in rain; radar is less angularly accurate but remains reliable in nearly all weather conditions.
  • Data Shadows: The paper provides a mathematical approach (Equation 1) to identify and remove "spurious data" or shadows caused by one sensor blocking another’s field of view.
  • Long-term Mapping: They propose the concepts of "self-aware localization" and "liability-aware long-term mapping," where the vehicle doesn't just use a map but actively assesses the map's reliability as the environment changes over months.

Key Results

  • Baseline Benchmarking: The authors successfully implemented and tested state-of-the-art lidar odometry methods (LOAM and LeGO-LOAM) on their dataset, providing a performance floor for future researchers.
  • Comparative Advantage: Compared to the famous KITTI dataset, this dataset offers more sensors (11 vs. 4), more diverse weather (snow/rain vs. clear), and a longer total distance for specific repeated routes (63.4km vs. 39.2km).
  • Centimeter-Level Ground Truth: By using GNSS-RTK with a fixed base station, they achieved centimeter-level positioning accuracy for their ground truth trajectories.

Limitations and Future Work

  • Processing Constraints: Data collection was performed using a CPU-only setup to ensure no data delay was introduced during recording. This means the raw data requires significant post-processing for tasks like motion compensation.
  • Hardware Synchronization: While software-level synchronization was achieved via ROS timestamps, the authors note that hardware-level synchronization (beyond the two Velodynes) was outside the scope of this initial work.
  • Future Directions: The authors intend to use this data to improve dynamic object prediction (predicting where pedestrians and cars will move) and to develop more robust "persistent mapping" techniques that can handle the "massive moving objects" found in dense urban odometry.

Summary updated

Abstract

The field of autonomous driving has grown tremendously over the past few years, along with the rapid progress in sensor technology. One of the major purposes of using sensors is to provide environment perception for vehicle understanding, learning and reasoning, and ultimately interacting with the environment. In this paper, we first introduce a multisensor platform allowing vehicle to perceive its surroundings and locate itself in a more efficient and accurate way. The platform integrates eleven heterogeneous sensors including various cameras and lidars, a radar, an IMU (Inertial Measurement Unit), and a GPS-RTK (Global Positioning System / Real-Time Kinematic), while exploits a ROS (Robot Operating System) based software to process the sensory data. Then, we present a new dataset (https://epan-utbm.github.io/utbm_robocar_dataset/) for autonomous driving captured many new research challenges (e.g. highly dynamic environment), and especially for long-term autonomy (e.g. creating and maintaining maps), collected with our instrumented vehicle, publicly available to the community.
BibTeX
@inproceedings{sun2020eu,
  author = {Kevin Li Sun and Zhi Yan and Li Sun and Tomás Krajník and Yassine Ruichek},
  title = {{EU Long-term Dataset with Multiple Sensors for Autonomous Driving}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2020},
  doi = {10.1109/iros45743.2020.9341406},
}

Loading related content...

Figure