IDD-3D: Indian Driving Dataset for 3D Unstructured Road Scenes

Shubham Dokania, A. H. Abdul Hafez, Anbumani Subramanian, Manmohan Chandraker, C. V. JawaharWACV 2023

TL;DR

This paper introduces IDD-3D, a novel multimodal dataset designed to capture the complexities of driving in unstructured Indian road environments. By providing 3D LiDAR and camera data with detailed annotations, IDD-3D aims to bridge the gap left by existing datasets that often focus on well-structured, Western driving scenarios, enabling better research into robust autonomous driving systems.

What Problem Does This Paper Address?

Autonomous driving systems rely heavily on large, annotated datasets to learn and adapt to diverse driving conditions. However, most existing datasets are collected in developed countries with structured traffic, clear regulations, and predictable object densities. This creates a significant gap when applying these systems to environments like India, characterized by highly unstructured roads, dense and varied traffic (including auto-rickshaws, hand carts, and animals), and complex inter-object interactions. Such scenarios pose a unique challenge for current models, which may struggle with the sheer diversity and unpredictability. IDD-3D directly addresses this by providing a dataset specifically curated to represent these challenging, real-world Indian driving conditions.

Methods

The core contribution of this paper is the IDD-3D dataset itself. The dataset comprises multimodal data from six RGB cameras and one Ouster OS1 LiDAR sensor, collected from a vehicle equipped with this sensor rig. The data processing pipeline includes sensor calibration and the preservation of raw sensor data in ROS bag format. A key aspect of data privacy is addressed through automated and manual blurring of faces and license plates.

For the dataset's analysis and benchmarking, the authors leverage established 3D object detection and tracking methods. Specifically, they employ SECOND (which uses voxelization and 3D convolutions) and CenterPoint (which focuses on object centers for better performance on smaller objects) for 3D object detection. For 3D object tracking, they utilize SimpleTrack, reporting metrics like AMOTA and AMOTP. The paper also discusses the potential benefits of pre-training models on existing large-scale datasets like KITTI and nuScenes before fine-tuning on IDD-3D, acknowledging that cross-dataset training might be challenging due to differing data distributions.

Datasets

The primary dataset introduced is IDD-3D, featuring 15.5k annotated LiDAR frames (with 12k for train-val) and over 223k 3D bounding boxes. The dataset includes 10 primary object categories and 7 additional miscellaneous categories, with annotations providing 9DoF data and instance IDs for tracking.

For pre-training and comparison, the authors reference and utilize data from:

  • KITTI: A well-known dataset for autonomous driving research, often used for benchmarking 3D object detection.
  • nuScenes: Another large-scale multimodal dataset for autonomous driving, known for its comprehensive sensor suite and diverse scenarios.
  • ONCE: A large-scale dataset for autonomous driving.

The paper highlights how IDD-3D differs from these by offering a higher traffic diversity and a wider distribution of object types and locations, particularly focusing on unstructured environments.

Key Insights and Ideas

  • The "Unstructured" Challenge is Real: The paper strongly emphasizes that existing datasets, while large, often fail to capture the true complexity of driving in many parts of the world, particularly in India. The sheer density, variety of object types (auto-rickshaws, hand carts, animals), and chaotic interactions are not well-represented elsewhere.
  • LiDAR is Crucial for 3D Perception in Chaos: The authors highlight the utility of 3D LiDAR sensors for better scene perception in these unstructured and "sporadically chaotic" traffic conditions, suggesting that multimodal sensor fusion is key.
  • Close-Range Dynamics Matter: IDD-3D's data distribution shows a significant number of objects close to the ego-vehicle. This is attributed to occlusions caused by dense traffic and is crucial for safety-critical split-second decision-making.
  • Diversity Breeds Robustness: The paper argues that datasets like IDD-3D, with their inherent diversity, are essential for developing autonomous systems that can generalize across different geographical locations and driving conditions.

Key Results

  • Dataset Size: IDD-3D contains 15.5k annotated LiDAR frames with over 223k 3D bounding boxes across 17 categories.
  • Object Density: The dataset exhibits higher object densities per frame compared to datasets like KITTI, with up to 50-60 objects in close proximity.
  • Performance Benchmarks: The paper presents experimental results for 3D object detection using CenterPoint, SECOND, and PointPillars on IDD-3D. CenterPoint generally shows better performance, likely due to its direct handling of point clouds.
  • Tracking Performance: For 3D object tracking, the authors report an overall AMOTA score of 0.472 on IDD-3D, noting that this is lower than scores achieved on datasets like nuScenes (0.668), underscoring the dataset's complexity. The Pedestrian class, in particular, shows lower tracking scores, indicating challenging motion patterns.

Limitations and Future Work

  • Data Collection Limitations: The data was collected during daytime sessions only, limiting its applicability to nighttime or adverse weather conditions.
  • Generalizability of Pre-training: While pre-training on other datasets is explored, the paper notes that cross-dataset training might not be straightforward due to significant differences in data distribution and features.
  • Future Work: The authors envision extending the IDD-3D dataset's applications to a wider range of tasks beyond object detection and tracking, including surveillance, road-safety analysis, and traffic quality assessment. They also aim for future releases to further enhance the dataset's utility.

Summary updated

BibTeX
@inproceedings{dokania2023idd,
  author = {Shubham Dokania and A. H. Abdul Hafez and Anbumani Subramanian and Manmohan Chandraker and C. V. Jawahar},
  title = {{IDD-3D: Indian Driving Dataset for 3D Unstructured Road Scenes}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2023},
}
Figure