Abstract
Millimeter wave (mmWave) radars are robust in extreme weather but suffer from sparsity and noise, limiting their use in micro aerial vehicle (MAV) navigation. This paper presents a cross-modal diffusion model that predicts dense, LiDAR-like point clouds from raw radar data. By integrating consistency model distillation, we achieve an 80x inference speedup, enabling real-time performance on resource-constrained MAVs. Extensive evaluations on public and custom datasets demonstrate superior accuracy and generalization.
Introduction
Millimeter-wave (mmWave) radars are essential for "all-weather" robotics, maintaining visibility in fog, smoke, and dust where LiDAR and cameras fail. However, traditional signal processing (e.g., CFAR) yields sparse, noisy point clouds unsuitable for complex obstacle avoidance or SLAM. While deep learning has attempted to upsample radar data, existing GAN or U-Net approaches often lack structural fidelity or fail in unseen environments. Furthermore, high-fidelity generative models like Diffusion Models are typically too computationally intensive for real-time MAV navigation.
Methodology
We frame radar-to-LiDAR translation as an image restoration task, drawing inspiration from low-resolution and noisy image denoising.
Data Representation and Framework
Raw radar signals are converted into Range-Azimuth Heatmaps (RAH), while LiDAR ground truth is projected into Bird’s Eye View (BEV) images. We employ a diffusion model based on the EDM framework to map the impaired radar domain to the dense LiDAR domain. During training, the model learns to "hallucinate" the density of LiDAR by using it as a ground-truth supervisor.
Loss Functions and Acceleration
To prioritize structural integrity (e.g., straight walls), we combine Mean Squared Error (MSE) with Learned Perceptual Image Patch Similarity (LPIPS). To overcome the latency of iterative diffusion, we distill the model into a Consistency Model. This allows the network to generate a final prediction from noise in a single step, drastically reducing inference time without significant loss in visual quality.
Experimental Evaluation
We validated our approach using the ColoRadar dataset (52 sequences across mines, labs, and outdoors) and a self-made dataset collected with a custom handheld rig featuring an Nvidia Jetson Orin NX, Livox Mid-360 LiDAR, and a TI-AWR1843BOOST radar.
Performance and Generalization
Our method consistently outperformed baselines like OS-CFAR, RadarHD, and RPDNet in Chamfer Distance (CD) and Hausdorff Distance (HD). Notably, the model generalized well to entirely unseen sensors and environments, maintaining high structural accuracy where traditional methods failed.
Real-Time Deployment
On the Nvidia Jetson Orin NX, the consistency model achieved an inference time of 181.6 ms (~5 Hz). This represents an 80x speedup over the standard iterative diffusion process, making it viable for real-time MAV navigation.
Conclusion and Future Work
This work demonstrates that cross-modal diffusion, combined with consistency distillation, can bridge the gap between low-cost radar and high-fidelity LiDAR perception. Future research will focus on extending the model to 3D point cloud generation, handling dynamic obstacles, and improving performance on low-reflectivity materials like vegetation.
Summary updated