Abstract
LiDAR-based 3D object detection often relies on aggregating multiple consecutive frames to overcome the sparsity of single-sweep point clouds. However, standard "fixed aggregation" methods face a fundamental trade-off: while more frames improve stationary object detection, they cause motion blur and misalignment for fast-moving objects. This paper introduces VADet (Variable Aggregation Detection), a method that adaptively chooses the optimal number of frames to aggregate based on an object's individual speed and point density. VADet achieves state-of-the-art performance on the Waymo Open Dataset while remaining architecture-agnostic and computationally efficient.
The Fixed Aggregation Trade-off
In autonomous driving, LiDAR sensors provide a sparse representation of the environment. Aggregating multiple frames helps fill in geometric details for stationary objects. However, for fast-moving vehicles, excessive aggregation leads to "smearing," where the point cloud becomes distorted and misaligned with the object's true bounding box.
As shown in the performance analysis, stationary objects see a steady increase in Average Precision (AP) as more frames are added. In contrast, fast-moving objects peak early and then experience a sharp decline in accuracy as motion-induced noise overwhelms the geometric benefits of additional points.
Methodology
VADet moves away from the "one-size-fits-all" approach through a two-stage strategy:
- Random Aggregation Training (RAT): To avoid the cost of training separate models for every possible frame count, VADet utilizes RAT. A single detector is trained where the number of aggregated frames varies randomly for each scene. This ensures the model is robust to varying input densities and allows for fair evaluation across different frame counts.
- Variable Aggregation Function ($\eta$): The core of the system is a lookup table that maps an object's estimated speed and point density to an empirically optimal number of frames.
- Per-Object Input Construction: VADet uses a constant velocity model to predict an object's prior locations. It then crops and aggregates the specific number of frames required for that object's region, providing a "customized" point cloud to the detector.
Key Results and Evaluation
The researchers evaluated VADet on the Waymo Open Dataset. By integrating VADet with the VoxelNeXt architecture, the team achieved 76.6 L2 AP on the validation set and 79.8 L2 AP on the test set, outperforming complex two-stage and multi-modal detectors.
Qualitative results highlight VADet's ability to maintain clear object boundaries. While 16-frame aggregation creates significant "tails" behind moving vehicles, VADet's adaptive approach provides enough density for detection without the distorting effects of motion blur. Furthermore, VADet is highly efficient, adding only ~50ms of overhead, making it far more practical for real-time applications than high-latency attention-based fusion methods.
Conclusion
VADet demonstrates that optimizing the input data through adaptive aggregation is a powerful alternative to increasing model complexity. By accounting for both speed and point density, VADet provides a consistent performance boost across various base architectures (CenterPoint, VoxelNeXt, DSVT). Future work may involve replacing the empirical lookup table with a learned function to map features directly to optimal aggregation strategies.
Summary updated