Abstract
LiDAR sensors are widely used for 3D object detection in various mobile robotics applications. LiDAR sensors continuously generate point cloud data in real-time. Conventional 3D object detectors detect objects using a set of points acquired over a fixed duration. However, recent studies have shown that the performance of object detection can be further enhanced by utilizing spatio-temporal information obtained from point cloud sequences. In this paper, we propose a new 3D object detector, named D-Align, which can effectively produce strong bird's-eye-view (BEV) features by aligning and aggregating the features obtained from a sequence of point sets. The proposed method includes a novel dual-query co-attention network that uses two types of queries, including target query set (T-QS) and support query set (S-QS), to update the features of target and support frames, respectively. D-Align aligns S-QS to T-QS based on the temporal context features extracted from the adjacent feature maps and then aggregates S-QS with T-QS using a gated attention mechanism. The dual queries are updated through multiple attention layers to progressively enhance the target frame features used to produce the detection results. Our experiments on the nuScenes dataset show that the proposed D-Align method greatly improved the performance of a single frame-based baseline method and significantly outperformed the latest 3D object detectors.
Introduction: The Challenge of Temporal Misalignment
LiDAR sensors provide essential 3D spatial data for autonomous systems, but individual sweeps are often sparse. A common strategy to increase point density is to merge consecutive sweeps into a single large point cloud. However, this approach faces a "diminishing returns" problem. As more frames are added, moving objects like vehicles or pedestrians create a "ghosting" or "smearing" effect due to their change in position over time. This temporal misalignment introduces noise that hinders a detector's ability to accurately estimate an object's precise location and velocity.
Methodology: Dual-Query Co-attention Network (D-Align)
D-Align addresses these challenges by moving beyond simple frame stacking. Instead, it intelligently aligns and aggregates features using a Dual-Query Co-attention Network (DUCANet). The system maintains two distinct query sets: the Target Query Set (T-QS) for the current frame and the Support Query Set (S-QS) for historical frames.
The architecture refines these queries through two primary modules:
- Inter-frame Deformable Alignment Network (IDANet): This module acts as the alignment engine. It utilizes Feature Pyramid Motion Extraction (FPMNet) to estimate motion between adjacent frames. By employing deformable attention guided by this motion context, it warps support features to align perfectly with the target frame.
- Inter-frame Gated Aggregation Network (IGANet): Once aligned, support features are merged into the target features. A gated mechanism learns to filter these features, prioritizing useful information while ignoring noise caused by occlusions or errors.
These steps are repeated across multiple layers ($L=3$) to iteratively sharpen the Bird’s-Eye-View (BEV) features before they reach the 3D object detection head.
Experimental Results
The researchers evaluated D-Align on the nuScenes dataset, a benchmark featuring complex urban environments with 1.4 million 3D bounding box annotations. D-Align was tested against two baselines: PointPillars and CenterPoint.
- Performance Gains: D-Align-CP (CenterPoint-based) achieved a state-of-the-art 70.2 NDS (nuScenes Detection Score) and 64.0 mAP on the test set, outperforming other multi-frame models like STGNN. The PointPillars-based version (D-Align-PP) saw a massive jump in NDS from 45.3% to 63.1% compared to its single-frame counterpart.
- Robustness to Sparsity: The model proved exceptionally effective for sparse object classes like motorcycles and traffic cones. By correctly aligning historical data, it effectively "densifies" the representation of these objects.
- Ablation Study: Experiments confirmed that the explicit motion modeling in FPMNet is critical; removing it led to a significant performance degradation, proving that motion context is a key ingredient for multi-frame success.
Conclusion and Future Work
D-Align demonstrates that explicit motion-guided alignment is a powerful tool for multi-frame 3D detection. While it significantly outperforms existing methods, limitations remain regarding the temporal horizon. Performance currently peaks at 3 frames (approx. 1.5 seconds), suggesting difficulty in maintaining coherence over longer durations where dynamic changes become too complex. Future research will focus on more efficient attention mechanisms to handle longer sequences and reduce memory overhead, further pushing the boundaries of temporal perception in autonomous driving.
Summary updated