Abstract
In this paper, we present LiRaNet, a novel end-to-end trajectory prediction method which utilizes radar sensor information along with widely used lidar and high definition (HD) maps. Automotive radar provides rich, complementary information, allowing for longer range vehicle detection as well as instantaneous radial velocity measurements. However, there are factors that make the fusion of lidar and radar information challenging, such as the relatively low angular resolution of radar measurements, their sparsity and the lack of exact time synchronization with lidar. To overcome these challenges, we propose an efficient spatio-temporal radar feature extraction scheme which achieves state-of-the-art performance on multiple large-scale datasets. Further, by incorporating radar information, we show a 52% reduction in prediction error for objects with high acceleration and a 16% reduction in prediction error for objects at longer range.
Introduction
Accurate motion prediction of surrounding actors is a cornerstone of safe autonomous driving. While lidar-based systems have become the standard due to their high-resolution 3D point clouds, they suffer from inherent limitations. Lidar primarily provides positional data, requiring multiple frames to estimate velocity and acceleration, which introduces dangerous latency in dynamic scenarios. Furthermore, lidar point density degrades significantly at long ranges and under occlusion.
Radar offers a compelling solution by providing instantaneous radial velocity via the Doppler effect and maintaining performance at greater distances. However, radar data is notoriously noisy and sparse, with poor angular resolution compared to lidar. LiRaNet addresses these challenges by effectively fusing radar, lidar, and HD maps in an end-to-end framework.
Methodology
The core of LiRaNet is a spatio-temporal feature learning scheme designed to handle the irregular nature of radar data.
Spatio-Temporal Radar Feature Learning
Unlike lidar points which can be easily projected onto a grid, radar points are sparse and unordered. LiRaNet employs parametric continuous convolutions to map these points onto a Bird’s Eye View (BEV) grid. By processing a temporal sequence of radar sweeps, the model can recover the full 2D velocity and yaw rate of actors, effectively bypassing the latency associated with lidar-only dynamics estimation.
Multi-Sensor Fusion
The architecture integrates features from three distinct sources: radar, lidar (voxelized occupancy grids), and HD maps (rasterized layers). These features are concatenated and processed through a multi-scale fusion backbone. The final prediction head outputs multiple probabilistic trajectories to account for multi-modal future behaviors.
Experimental Results
LiRaNet was evaluated on the public nuScenes dataset and Uber ATG’s large-scale X17k dataset. The inclusion of radar led to substantial performance gains, particularly in "edge case" scenarios where lidar is most vulnerable.
- High Dynamics: For objects undergoing rapid acceleration or deceleration, LiRaNet achieved a 52% reduction in prediction error. Radar's ability to sense speed changes instantly provides a "dynamics shortcut" that lidar-only models lack.
- Long-Range Performance: At distances between 70m and 100m, where lidar points are sparse, radar fusion resulted in a 16% error reduction.
- Real-Time Viability: The system maintains a low inference latency of approximately 40ms, making it suitable for real-world self-driving stacks.
Conclusion
LiRaNet demonstrates that radar is a powerful, yet often underutilized, sensor for motion prediction. By overcoming the challenges of radar noise and sparsity through spatio-temporal feature learning, the model significantly improves safety in high-acceleration and long-range scenarios. Future work will explore the use of raw radar tensors (Range-Azimuth-Doppler) to further enhance feature extraction and prediction accuracy.
Summary updated