L2RDaS: Synthesizing 4D Radar Tensors for Model Generalization via Dataset Expansion

Woo-Jin Jung, Dong-Hee Paek, Seung-Hyun Kong

TL;DR

L2RDaS is a novel framework that synthesizes high-fidelity 4D radar tensors from existing LiDAR datasets, addressing the severe scarcity of public 4D radar data. By utilizing a modified U-Net architecture and a specialized Object Information Supplement (OBIS) module, the framework enables massive dataset expansion and ground-truth augmentation, leading to significant performance gains in 4D radar-based object detection.

What Problem Does This Paper Address?

While LiDAR and cameras are staples of autonomous driving, they fail in adverse weather like heavy fog or snow. 4D radar is the robust alternative, providing range, azimuth, elevation, and Doppler information regardless of lighting or precipitation.

However, the field faces a "data drought." There are over 30 publicly available LiDAR datasets, but only five for 4D radar tensors. Most existing radar datasets provide sparse point clouds generated via traditional Constant False Alarm Rate (CFAR) filtering. While CFAR removes noise, it also strips away critical spatial information, resulting in sparse data that makes precise object boundary detection difficult. To move toward more accurate perception, researchers are shifting to 4D radar tensors (dense grids of reflected power), but the lack of diverse training data severely limits model generalization.

Methods

The authors propose L2RDaS (LiDAR-to-4D Radar data Synthesis), a framework based on conditional Generative Adversarial Networks (cGAN).

The L2RDaS framework overview, showing the Generator and the OBIS module.

Key technical components include:

  • Redesigned Generator: Based on the pix2pixHD architecture but adapted for 3D spatial data. It uses a sparse 3D convolution encoder to efficiently process sparse LiDAR voxels and a dense 3D convolution decoder to reconstruct the full, dense radar tensor.
  • OBIS (Object Information Supplement) Module: This is the core innovation. LiDAR is precise, but radar measurements often exhibit positional deviations and lower resolution. OBIS enriches the LiDAR input by:
  • Adding Object Boundary Points: Uniformly sampled points along 3D bounding box edges to provide explicit geometric cues.
  • Adding Gaussian-Distributed Points: Spherical "spatial anchors" around object centers that help the generator produce realistic measurement distributions and distinguish between object classes.
  • Adversarial Training: The model is trained using a combination of conditional adversarial loss, feature matching loss, and L1 reconstruction loss to ensure the synthesized tensors are indistinguishable from real sensor data.

Datasets

The primary dataset for training and evaluation is K-Radar, which provides synchronized 4D radar tensors and LiDAR.

To demonstrate the "Dataset Expansion" capability, the authors synthesized 4D radar tensors for four major autonomous driving datasets that originally lacked them:

  • KITTI
  • nuScenes
  • VoD (View-of-Delft)
  • Dual-Radar
Qualitative results showing synthesized radar tensors across various datasets.

Key Insights and Ideas

  • Bridging the Resolution Gap: The authors quantified that radar object centers shift by an average of 1.215 meters compared to LiDAR's 0.102 meters. The OBIS module is critical because it provides the "fuzziness" and geometric context the generator needs to bridge this 10x difference in precision.
  • Dense vs. Sparse Decoders: An ablation study revealed that using dense convolution in the decoder is superior to sparse convolution for this task. Dense layers ensure complete spatial coverage, which is essential for producing the continuous power distributions found in real radar tensors.
  • Synthetic Data as a Generalizer: Training on a mix of real K-Radar data and synthetic data from other domains (like nuScenes or KITTI) consistently improved performance on the real K-Radar test set. This proves that L2RDaS captures meaningful, domain-invariant radar characteristics.

Key Results

The framework was evaluated using three different detection models: RTNH, RadarPillar-Net, and RPFA-Net.

  • Synthesis Quality: The synthesized tensors achieved a PSNR of 31.01 dB and an SSIM of 0.90, indicating high structural and pixel-wise similarity to real ground-truth tensors.
  • Dataset Expansion: Adding synthesized data from external datasets to the training set resulted in an average increase of 4.25% in $AP_{BEV}$ and 2.87% in $AP_{3D}$.
  • Ground-Truth Augmentation (GT-Aug): By performing "cut-and-paste" augmentation on LiDAR and then synthesizing the radar equivalent, the authors achieved further gains of 3.75% in $AP_{BEV}$ and 4.03% in $AP_{3D}$.
Ablation study showing how the OBIS module improves the realism of synthesized object reflections.

Limitations and Future Work

The primary limitation is the lack of Doppler information. Because the synthesis is based on single-frame LiDAR, it lacks the temporal cues necessary to estimate the relative velocity of objects.

Future work aims to:

  1. Leverage temporal LiDAR sequences to synthesize Doppler information.
  2. Improve the alignment between synthesized and real radar distributions to further reduce the "sim-to-real" gap.

Summary updated

Abstract

4-dimensional (4D) radar is increasingly adopted in autonomous driving for perception tasks, owing to its robustness under adverse weather conditions. To better utilize the spatial information inherent in 4D radar data, recent deep learning methods have transitioned from using sparse point cloud to 4D radar tensors. However, the scarcity of publicly available 4D radar tensor datasets limits model generalization across diverse driving scenarios. Previous methods addressed this by synthesizing radar data, but the outputs did not fully exploit the spatial information characteristic of 4D radar. To overcome these limitations, we propose LiDAR-to-4D radar data synthesis (L2RDaS), a framework that synthesizes spatially informative 4D radar tensors from LiDAR data available in existing autonomous driving datasets. L2RDaS integrates a modified U-Net architecture to effectively capture spatial information and an object information supplement (OBIS) module to enhance reflection fidelity. This framework enables the synthesis of radar tensors across diverse driving scenarios without additional sensor deployment or data collection. L2RDaS improves model generalization by expanding real datasets with synthetic radar tensors, achieving an average increase of 4.25\% in ${{AP}_{BEV}}$ and 2.87\% in ${{AP}_{3D}}$ across three detection models. Additionally, L2RDaS supports ground-truth augmentation (GT-Aug) by embedding annotated objects into LiDAR data and synthesizing them into radar tensors, resulting in further average increases of 3.75\% in ${{AP}_{BEV}}$ and 4.03\% in ${{AP}_{3D}}$. The implementation will be available at https://github.com/kaist-avelab/K-Radar.
BibTeX
@misc{jung2025l,
  author = {Woo-Jin Jung and Dong-Hee Paek and Seung-Hyun Kong},
  title = {{L2RDaS: Synthesizing 4D Radar Tensors for Model Generalization via Dataset Expansion}},
  year = {2025},
  eprint = {2503.03637},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2503.03637},
}
Figure