4D Radar Ground Truth Augmentation with LiDAR-to-4D Radar Data Synthesis

Woo-Jin Jung, Dong-Hee Paek, Seung-Hyun Kong

Abstract

Ground truth augmentation (GT-Aug) is a common method for LiDAR-based object detection, as it enhances object density by leveraging ground truth bounding boxes (GT bboxes). However, directly applying GT-Aug to 4D Radar tensor data overlooks important measurements outside the GT bboxes—such as sidelobes—leading to synthetic distributions that deviate from real-world 4D Radar data. To address this limitation, we propose 4D Radar Ground Truth Augmentation (4DR GT-Aug). Our approach first augments LiDAR data and then converts it to 4D Radar data via a LiDAR-to-4D Radar data synthesis (L2RDaS) module, which explicitly accounts for measurements both inside and outside GT bboxes. In doing so, it produces 4D Radar data distributions that more closely resemble real-world measurements, thereby improving object detection accuracy. Experiments on the K-Radar dataset show that the proposed method achieves improved performance compared to conventional GT-Aug in object detection for 4D Radar.

Introduction

4D radar is becoming a cornerstone of autonomous driving perception due to its resilience in adverse weather conditions like fog, rain, and snow. However, the field faces a significant data bottleneck. While LiDAR datasets are abundant, high-quality datasets providing dense 4D radar tensors are scarce. Unlike sparse point clouds, 4D radar tensors preserve critical spatial information but are difficult to augment using standard techniques. Conventional "Ground Truth Augmentation" (GT-Aug), which involves pasting objects into scenes, fails to capture the complex physical properties of radar reflections, such as sidelobes and environmental clutter.

Overview of L2RDaS applications in dataset expansion and ground truth augmentation.

To bridge this gap, we introduce L2RDaS (LiDAR-to-4D Radar data Synthesis), a framework designed to generate realistic 4D radar tensors from existing LiDAR data. This enables both the expansion of radar-absent datasets and more effective ground-truth augmentation.

Methodology: The L2RDaS Framework

The core of our approach is a generative framework based on a Conditional Generative Adversarial Network (cGAN).

L2RDaS Generator

The generator architecture is inspired by pix2pixHD, utilizing a modified U-Net structure. It employs sparse 3D convolutions in the encoder to process high-resolution but sparse LiDAR voxels efficiently. Conversely, the decoder uses dense 3D convolutions to reconstruct the dense power distribution of the 4D radar tensor. This design choice is critical, as dense layers are superior at "filling in" spatial gaps to produce realistic radar "blobs."

Object Information Supplement (OBIS) Module

A significant challenge in synthesis is the resolution mismatch between LiDAR and radar. Radar object centers often deviate from LiDAR centers due to lower resolution. To address this, the OBIS module enriches the LiDAR input before it reaches the generator:

  • Boundary Point Addition: Uniformly samples points along 3D bounding box edges to provide geometric anchors for object localization.
  • Gaussian-Distributed Point Injection: Adds auxiliary points in a 3D Gaussian distribution around object centers, guiding the generator to produce realistic reflection power and spatial extent.
The L2RDaS framework, highlighting the Object Information Supplement (OBIS) module and the generator architecture.

4D Radar GT-Aug

By embedding annotated objects into LiDAR data before synthesis, L2RDaS allows the generator to "render" these objects into the radar tensor. This process naturally incorporates realistic environmental clutter and physical artifacts that manual pasting overlooks.

Experimental Results

We evaluated L2RDaS using the K-Radar dataset and demonstrated its generalization by synthesizing radar tensors for four major LiDAR datasets: KITTI, nuScenes, VoD, and Dual-Radar.

Dataset Expansion and Performance

Training models on synthesized tensors from external datasets alongside real K-Radar data led to significant improvements. We observed an average increase of 4.25% in $AP_{BEV}$ and 2.87% in $AP_{3D}$ across multiple detection models (RTNH, RadarPillar-Net, and RPFA-Net). Furthermore, L2RDaS-powered GT-Aug provided additional gains of 3.75% in $AP_{BEV}$ and 4.03% in $AP_{3D}$.

Synthesis Quality

The high fidelity of our synthesized tensors was confirmed by quantitative metrics, achieving a Peak Signal-to-Noise Ratio (PSNR) of 31.006 dB and a Structural Similarity Index (SSIM) of 0.897 compared to real-world radar measurements.

Conclusion

L2RDaS effectively addresses the scarcity of 4D radar data by leveraging abundant LiDAR resources. By incorporating the OBIS module and a hybrid sparse-dense generator, the framework produces high-fidelity radar tensors that significantly enhance object detection performance. Future work will focus on incorporating temporal LiDAR sequences to synthesize Doppler (velocity) information, further closing the sim-to-real gap.

Summary updated

Abstract

Ground truth augmentation (GT-Aug) is a common method for LiDAR-based object detection, as it enhances object density by leveraging ground truth bounding boxes (GT bboxes). However, directly applying GT-Aug to 4D Radar tensor data overlooks important measurements outside the GT bboxes-such as sidelobes-leading to synthetic distributions that deviate from real-world 4D Radar data. To address this limitation, we propose 4D Radar Ground Truth Augmentation (4DR GT-Aug). Our approach first augments LiDAR data and then converts it to 4D Radar data via a LiDAR-to-4D Radar data synthesis (L2RDaS) module, which explicitly accounts for measurements both inside and outside GT bboxes. In doing so, it produces 4D Radar data distributions that more closely resemble real-world measurements, thereby improving object detection accuracy. Experiments on the K-Radar dataset show that the proposed method achieves improved performance compared to conventional GT-Aug in object detection for 4D Radar. The implementation code is available at https://github.com/kaist-avelab/K-Radar.
BibTeX
@misc{jung2025d,
  author = {Woo-Jin Jung and Dong-Hee Paek and Seung-Hyun Kong},
  title = {{4D Radar Ground Truth Augmentation with LiDAR-to-4D Radar Data Synthesis}},
  year = {2025},
  eprint = {2503.03637},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2503.03637},
}
Figure