Abstract
Ground truth augmentation (GT-Aug) is a common method for LiDAR-based object detection, as it enhances object density by leveraging ground truth bounding boxes (GT bboxes). However, directly applying GT-Aug to 4D Radar tensor data overlooks important measurements outside the GT bboxes—such as sidelobes—leading to synthetic distributions that deviate from real-world 4D Radar data. To address this limitation, we propose 4D Radar Ground Truth Augmentation (4DR GT-Aug). Our approach first augments LiDAR data and then converts it to 4D Radar data via a LiDAR-to-4D Radar data synthesis (L2RDaS) module, which explicitly accounts for measurements both inside and outside GT bboxes. In doing so, it produces 4D Radar data distributions that more closely resemble real-world measurements, thereby improving object detection accuracy. Experiments on the K-Radar dataset show that the proposed method achieves improved performance compared to conventional GT-Aug in object detection for 4D Radar.
Introduction
4D radar is becoming a cornerstone of autonomous driving perception due to its resilience in adverse weather conditions like fog, rain, and snow. However, the field faces a significant data bottleneck. While LiDAR datasets are abundant, high-quality datasets providing dense 4D radar tensors are scarce. Unlike sparse point clouds, 4D radar tensors preserve critical spatial information but are difficult to augment using standard techniques. Conventional "Ground Truth Augmentation" (GT-Aug), which involves pasting objects into scenes, fails to capture the complex physical properties of radar reflections, such as sidelobes and environmental clutter.
To bridge this gap, we introduce L2RDaS (LiDAR-to-4D Radar data Synthesis), a framework designed to generate realistic 4D radar tensors from existing LiDAR data. This enables both the expansion of radar-absent datasets and more effective ground-truth augmentation.
Methodology: The L2RDaS Framework
The core of our approach is a generative framework based on a Conditional Generative Adversarial Network (cGAN).
L2RDaS Generator
The generator architecture is inspired by pix2pixHD, utilizing a modified U-Net structure. It employs sparse 3D convolutions in the encoder to process high-resolution but sparse LiDAR voxels efficiently. Conversely, the decoder uses dense 3D convolutions to reconstruct the dense power distribution of the 4D radar tensor. This design choice is critical, as dense layers are superior at "filling in" spatial gaps to produce realistic radar "blobs."
Object Information Supplement (OBIS) Module
A significant challenge in synthesis is the resolution mismatch between LiDAR and radar. Radar object centers often deviate from LiDAR centers due to lower resolution. To address this, the OBIS module enriches the LiDAR input before it reaches the generator:
- Boundary Point Addition: Uniformly samples points along 3D bounding box edges to provide geometric anchors for object localization.
- Gaussian-Distributed Point Injection: Adds auxiliary points in a 3D Gaussian distribution around object centers, guiding the generator to produce realistic reflection power and spatial extent.
4D Radar GT-Aug
By embedding annotated objects into LiDAR data before synthesis, L2RDaS allows the generator to "render" these objects into the radar tensor. This process naturally incorporates realistic environmental clutter and physical artifacts that manual pasting overlooks.
Experimental Results
We evaluated L2RDaS using the K-Radar dataset and demonstrated its generalization by synthesizing radar tensors for four major LiDAR datasets: KITTI, nuScenes, VoD, and Dual-Radar.
Dataset Expansion and Performance
Training models on synthesized tensors from external datasets alongside real K-Radar data led to significant improvements. We observed an average increase of 4.25% in $AP_{BEV}$ and 2.87% in $AP_{3D}$ across multiple detection models (RTNH, RadarPillar-Net, and RPFA-Net). Furthermore, L2RDaS-powered GT-Aug provided additional gains of 3.75% in $AP_{BEV}$ and 4.03% in $AP_{3D}$.
Synthesis Quality
The high fidelity of our synthesized tensors was confirmed by quantitative metrics, achieving a Peak Signal-to-Noise Ratio (PSNR) of 31.006 dB and a Structural Similarity Index (SSIM) of 0.897 compared to real-world radar measurements.
Conclusion
L2RDaS effectively addresses the scarcity of 4D radar data by leveraging abundant LiDAR resources. By incorporating the OBIS module and a hybrid sparse-dense generator, the framework produces high-fidelity radar tensors that significantly enhance object detection performance. Future work will focus on incorporating temporal LiDAR sequences to synthesize Doppler (velocity) information, further closing the sim-to-real gap.
Summary updated