EU Long-term Dataset with Multiple Sensors for Autonomous Driving

Zhi Yan, Li Sun, Tomáš Krajník, Yassine Ruichek

Abstract

The field of autonomous driving has grown tremendously over the past few years, along with the rapid progress in sensor technology. One of the major purposes of using sensors is to provide environment perception for vehicle understanding, learning and reasoning, and ultimately interacting with the environment. In this paper, we first introduce a multisensor platform allowing vehicle to perceive its surroundings and locate itself in a more efficient and accurate way. The platform integrates eleven heterogeneous sensors including various cameras and lidars, a radar, an IMU (Inertial Measurement Unit), and a GPS-RTK (Global Positioning System / Real-Time Kinematic), while exploits a ROS (Robot Operating System) based software to process the sensory data. Then, we present a new dataset for autonomous driving captured many new research challenges (e.g. highly dynamic environment), and especially for long-term autonomy (e.g. creating and maintaining maps), collected with our instrumented vehicle, publicly available to the community.

Introduction

Most autonomous driving (AD) datasets are "snapshots" in time—they capture a few hours of driving in fair weather. However, for a vehicle to be truly autonomous, it must achieve long-term autonomy. This requires navigating the same streets whether it’s a sunny summer afternoon or a snowy winter night.

The authors identify a gap in existing data: a lack of high-frequency, multisensor recordings that capture periodic environmental changes (seasonal/daily) and complex traffic scenarios like roundabouts. They also address the engineering challenge of building a platform that synchronizes eleven heterogeneous sensors to provide a seamless 360-degree view of the environment.

System Architecture and Methodology

The authors developed an instrumented vehicle using a ROS-based (Robot Operating System) software architecture. The platform integrates 11 sensors, including two Velodyne 32-layer lidars, two stereo cameras, two fisheye cameras, a 2D lidar for road markings, radar, IMU, and GNSS-RTK.

Sensor placement and hardware integration on the experimental vehicle.

Key technical highlights include:

  • Interference Mitigation: To use two Velodyne lidars side-by-side without them "blinding" each other, the team utilized a hardware phase-locking feature. This synchronized the laser firings so they wouldn't overlap in the same spatial region at the same time.
  • Calibration: Cameras were calibrated using standard chessboard patterns, while lidar-to-camera extrinsic parameters were estimated by minimizing voxel-wise $L_2$ distances between point clouds and image features.
  • Privacy Compliance: In accordance with EU GDPR, deep learning-based methods were used to post-process images, blurring faces and license plates before public release.

The EU Long-term Dataset (UTBM Robocar Dataset)

The primary contribution is a comprehensive dataset featuring:

  • Scale: 63.4 km of total driving data recorded in Montbéliard, France.
  • Temporal Diversity: 11 rounds of a specific 5km loop recorded at different times (day/night) and seasons (summer/winter/snow).
  • Complex Maneuvers: A dedicated subset focusing on roundabouts (10 different ones), which are notoriously difficult for AD systems to predict.
  • Format: Provided in rosbag format for easy integration with existing robotics pipelines.

Key Insights: Sensor Complementarity and Long-term Mapping

The paper frames AD as a continuous "See-Think-Act" loop where perception and localization are the most critical components for safety. The authors argue against relying on a single sensor type, emphasizing sensor complementarity. For example, while 3D lidar is accurate, it struggles in rain; radar is less angularly accurate but remains reliable in nearly all weather conditions.

Horizontal field of view (FOV) coverage for the heterogeneous sensor suite.

To handle the multi-sensor setup, the paper provides a mathematical approach to identify and remove "spurious data" or shadows caused by one sensor blocking another’s field of view. Furthermore, they propose the concepts of "self-aware localization" and "liability-aware long-term mapping," where the vehicle actively assesses the map's reliability as the environment changes over months.

Benchmarking and Results

The authors successfully implemented and tested state-of-the-art lidar odometry methods (LOAM and LeGO-LOAM) on their dataset. Compared to the famous KITTI dataset, this dataset offers more sensors (11 vs. 4), more diverse weather (snow/rain vs. clear), and a longer total distance for specific repeated routes (63.4km vs. 39.2km). By using GNSS-RTK with a fixed base station, they achieved centimeter-level positioning accuracy for their ground truth trajectories.

Conclusion and Future Work

While software-level synchronization was achieved via ROS timestamps, the authors note that hardware-level synchronization (beyond the two Velodynes) remains a future goal. Data collection was performed using a CPU-only setup to ensure no recording delays, meaning raw data requires post-processing for motion compensation. Future work will focus on using this data to improve dynamic object prediction and to develop more robust "persistent mapping" techniques that can handle the "massive moving objects" found in dense urban environments.

Summary updated

Abstract

The field of autonomous driving has grown tremendously over the past few years, along with the rapid progress in sensor technology. One of the major purposes of using sensors is to provide environment perception for vehicle understanding, learning and reasoning, and ultimately interacting with the environment. In this paper, we first introduce a multisensor platform allowing vehicle to perceive its surroundings and locate itself in a more efficient and accurate way. The platform integrates eleven heterogeneous sensors including various cameras and lidars, a radar, an IMU (Inertial Measurement Unit), and a GPS-RTK (Global Positioning System / Real-Time Kinematic), while exploits a ROS (Robot Operating System) based software to process the sensory data. Then, we present a new dataset (https://epan-utbm.github.io/utbm_robocar_dataset/) for autonomous driving captured many new research challenges (e.g. highly dynamic environment), and especially for long-term autonomy (e.g. creating and maintaining maps), collected with our instrumented vehicle, publicly available to the community.
BibTeX
@inproceedings{yan2020eu,
  author = {Zhi Yan and Li Sun and Tomáš Krajník and Yassine Ruichek},
  title = {{EU Long-term Dataset with Multiple Sensors for Autonomous Driving}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2020},
  doi = {10.1109/iros45743.2020.9341406},
}
Figure