TL;DR
MS-DOTT is a new multi-modal dataset designed to push the boundaries of off-road autonomous navigation by bridging the gap between long-range environmental context and fine-grained, near-field hazard detection. Featuring a diverse sensor suite—including HDR cameras, thermal imaging, and high-density Time-of-Flight (ToF) sensors—it provides 3,000 meters of day and night trajectories to help researchers tackle the "unstructured" challenges of off-road traversability.
What Problem Does This Paper Address?
Navigating off-road is a significantly harder problem than urban driving. In a city, you have lanes, signs, and predictable surfaces. Off-road, a patch of "vegetation" might be traversable grass or a wheel-swallowing ditch hidden by weeds. Furthermore, lighting conditions in forests or fields are extreme, ranging from blinding glare to total darkness.
The authors identify three critical gaps in existing datasets:
- Semantic Ambiguity: Knowing a region is "grass" isn't enough; you need to know its slope and roughness to determine if it's actually traversable.
- Illumination Limits: Most datasets fail in low-light or high-glare scenarios.
- Scale Mismatch: Sensors often focus on either long-range mapping (LiDAR) or short-range vision, but rarely provide the high-resolution, near-field data needed to spot "micro-obstacles" like small rocks or roots right in front of the vehicle.
Methods
The researchers built a custom "Data Collection Bench" (shown in Figure 2) mounted on a Botsync Copernicus UGV.
The sensor suite is impressively comprehensive:
- Vision: Standard RGB (FLIR), HDR RGB (Lucid TDR054S) for high dynamic range, Thermal (Trimer LWIR) for night, and Stereo (ZED2).
- 3D Sensing: Long-range solid-state LiDAR (Livox HAP), mid-range LiDAR (Livox MID360), and a high-density Time-of-Flight (ToF) camera (Lucid Helios2+) for ultra-dense near-field detail.
- Additional: 4D RADAR (Continental ARS548) for weather-agnostic velocity sensing and an Xsens IMU/GNSS for localization.
To process this data, they developed custom ROS2 drivers and a CUDA-accelerated "point painting" algorithm to project image data onto 3D point clouds in real-time. For evaluation, they adapted SALON, a state-of-the-art traversability fusion method, swapping its standard vision backbone for AnyThermal to enable traversability estimation in total darkness.
Datasets
The primary contribution is the MS-DOTT (Multi-Sensor Dataset for Off-road Terrain Traversability).
- Scope: 11 trajectories (6 day, 5 night) covering 3000m in forest-like environments.
- Volume: 90 minutes of data, 432,000 camera frames, and 243,000 point clouds.
- Comparison: Unlike predecessors like RUGD or RELLIS-3D, MS-DOTT includes ToF and 4D Radar, and specifically targets the fusion of long-range and fine-grained short-range data across day and night.
Key Insights and Ideas
- HDR is Essential for Glare: Standard RGB cameras often lose all detail in high-contrast shadows or direct sunlight (glare). The HDR camera successfully captures obstacles that are "invisible" to standard sensors due to overexposure.
- The ToF "Micro-Obstacle" Advantage: Traditional LiDARs can be too sparse at very close ranges. The high-density ToF sensor captures small hazards like pebbles and roots that traditional 3D sensing might miss, providing a much more expressive local costmap.
- Thermal-LiDAR Fusion for Night Ops: By using thermal features instead of RGB, the authors show that traversability costmaps can remain accurate even when visual cameras are completely underexposed.
- Multi-Scale Sensing: The dataset proves that a single sensor isn't enough. Effective off-road navigation requires a "hierarchy" of sensors: ToF for the immediate path, mid-range LiDAR for local mapping, and long-range LiDAR for global path planning.
Key Results
The authors qualitatively demonstrated the utility of MS-DOTT by comparing costmaps generated from different sensor combinations.
As seen in Figure 3, in low-light conditions, the standard FLIR camera falsely marks clear areas as high-cost (red), while the HDR and Thermal sensors correctly identify the traversable terrain (yellow/green).
In Figure 4, the ToF sensor identifies near-field obstacles as high-cost (green box), whereas the more traditional HAP and MID360 LiDARs fail to resolve these close-in hazards (red boxes).
Limitations and Future Work
While comprehensive, the authors acknowledge several limitations:
- Field of View: The sensors are strictly forward-facing, lacking the 360-degree coverage found in some other recent datasets.
- Synchronization: The dataset relies on software-based synchronization (median offset of 8.7ms) rather than hardware-level triggering.
- Trajectory Length: Due to the massive data volume generated by nine sensors, trajectories are relatively short (~250m each), which limits their use for modeling high-speed vehicle dynamics.
- ToF Sensitivity: The ToF sensor, while excellent at night, sees a significant reduction in point cloud density under harsh direct sunlight.
Future work aims to address these by expanding the geographic diversity of the testing sites and implementing hardware-level time synchronization.
Summary updated