Get It for Free: Radar Segmentation Without Expert Labels and Its Application in Odometry and Localization

Siru Li, Ziyang Hong, Yushuai Chen, Liang Hu, Jiahu Qin

TL;DR

This paper introduces a novel weakly supervised method for semantic segmentation of radar data, leveraging LiDAR data for label generation and refinement. This approach significantly improves radar odometry and localization accuracy, especially in adverse weather conditions, outperforming existing methods.

What Problem Does This Paper Address?

Autonomous driving systems heavily rely on sensors like cameras and LiDAR for environmental perception. However, these sensors struggle in adverse weather conditions such as rain, snow, and fog due to occlusion. Radar, operating at a lower frequency, offers robust perception in these scenarios. Despite its advantages, radar data is often sparse and noisy, and crucially, lacks the rich semantic annotations available for camera and LiDAR data. This absence of semantic information hinders its full potential in tasks like simultaneous localization and mapping (SLAM), odometry, and localization. Existing methods for radar semantic segmentation either require manual annotation (expensive and time-consuming) or struggle with noise and adverse weather. This paper aims to bridge this gap by enabling accurate radar semantic segmentation using readily available LiDAR data as a supervisory signal, without requiring manual labeling.

Methods

The core of this work is a weakly supervised approach to radar semantic segmentation. The process involves several key steps:

  • LiDAR Label Generation: The authors first utilize a state-of-the-art LiDAR semantic segmentation model to generate initial semantic labels for LiDAR point clouds.
  • LiDAR Label Refinement: Recognizing that LiDAR-derived labels can still contain errors, they propose a novel refinement scheme. This scheme leverages structural features and distribution patterns within the LiDAR point cloud to correct misclassifications. Specifically, they use Singular Value Decomposition (SVD) to analyze the geometric structure of point clouds (e.g., distinguishing between planar building structures and more diffuse vegetation) and Axis-Aligned Bounding Boxes (AABB) to spatially enclose and correct vegetation segments.
  • Cross-Modal Supervision: The refined LiDAR semantic labels are then used as a supervisory signal to train a radar semantic segmentation model. This is a crucial step, as it allows the radar model to learn semantic understanding without direct manual annotation of radar data.
  • Downstream Task Integration: The generated radar semantic segmentation is then applied to two critical downstream tasks:
  • Radar Odometry: By identifying and retaining features from stable objects like buildings, the method improves the robustness of odometry estimation, especially in dynamic environments where moving objects can introduce errors. They also incorporate IMU data for improved rotation estimation.
  • Localization: Using OpenStreetMap (OSM) building data as a prior map, the semantic information from radar helps in more stable and accurate localization, particularly by focusing registration on reliable building features.

Datasets

The paper evaluates its approach on two publicly available datasets:

  • Boreas Dataset: This dataset includes Navtech CIR304-H radar, Velodyne Alpha-Prime 3D LiDAR, and IMU data. It is noted for its higher point cloud density and superior perception range from LiDAR.
  • Oxford Radar RobotCar Dataset: This dataset features Navtech CTS350x radar and two Velodyne HDL32 LiDARs. It is often used for odometry testing and collected in a more urban environment.

The authors do not introduce a new benchmark dataset but rather leverage existing ones to demonstrate the effectiveness of their method across different environments and sensor configurations.

Key Insights and Ideas

  • Leveraging LiDAR for Radar Supervision: The fundamental insight is that LiDAR, despite its weather limitations, can provide high-quality semantic labels that can be effectively transferred to radar, enabling robust radar segmentation without manual effort.
  • Importance of Label Refinement: Simply using raw LiDAR segmentation is insufficient. The proposed refinement step, which analyzes geometric structure and spatial distribution, is critical for improving the accuracy and reliability of the supervisory signals.
  • Semantic Information for Robustness: The semantic understanding of radar data is shown to be crucial for improving the robustness of odometry and localization. By focusing on stable features (like buildings) and filtering out dynamic or noisy elements, the system achieves more consistent performance, especially in challenging conditions.
  • All-Weather Performance: The method's ability to maintain stable segmentation and improve downstream tasks in adverse weather (rain, snow, fog) is a significant advantage over vision-based methods.

Key Results

  • Radar Semantic Segmentation: While specific IoU scores are provided (e.g., ~0.398 for building, ~0.297 for vehicle on Oxford dataset), the paper emphasizes that the generated semantic radar data leads to significant improvements in downstream tasks.
  • Radar Odometry: The proposed method achieved the first place in the radar odometry competition at the ICRA 2024 Radar in Robotics workshop. On the Boreas dataset, removing vehicles and using semantic information with IMU resulted in translation error reductions of up to 20.55% compared to prior methods.
  • Localization: The method demonstrated a significant improvement in average position error, with improvements of at least 30% on some sequences of the Boreas dataset when compared to the baseline method [10].

Limitations and Future Work

  • LiDAR Weather Dependency: The method's reliance on LiDAR for initial label generation means it is still indirectly affected by LiDAR's limitations in severe weather. While radar compensates for this in its own segmentation, the initial supervision quality can be impacted.
  • Lightweight Architecture: The paper notes that a lightweight architecture was adopted for real-time requirements, which can limit feature extraction capabilities and lead to suboptimal segmentation performance, particularly for complex scenes.
  • Vegetation Segmentation Challenges: Distinguishing between dense urban buildings and vegetation remains a challenge, and the IoU for vegetation is noted to be lower.
  • Future Work: The authors suggest further exploration into improving feature extraction capabilities and potentially extending the approach to other sensor modalities or more complex semantic categories. The potential for using the refined radar semantic segmentation for other SLAM-related tasks is also implied.

Summary updated

BibTeX
@article{li2025get,
  author = {Siru Li and Ziyang Hong and Yushuai Chen and Liang Hu and Jiahu Qin},
  title = {{Get It for Free: Radar Segmentation Without Expert Labels and Its Application in Odometry and Localization}},
  journal = {IEEE Robotics Autom. Lett.},
  year = {2025},
  doi = {10.1109/lra.2025.3536196},
}
Figure