VADet: Multi-Frame LiDAR 3D Object Detection Using Variable Aggregation

Chengjie Huang, Vahdat Abdelzad, Sean Sedwards, Krzysztof Czarnecki

Abstract

LiDAR-based 3D object detection often relies on aggregating multiple consecutive frames to overcome the sparsity of single-sweep point clouds. However, standard "fixed aggregation" methods face a fundamental trade-off: while more frames improve stationary object detection, they cause motion blur and misalignment for fast-moving objects. This paper introduces VADet (Variable Aggregation Detection), a method that adaptively chooses the optimal number of frames to aggregate based on an object's individual speed and point density. VADet achieves state-of-the-art performance on the Waymo Open Dataset while remaining architecture-agnostic and computationally efficient.

The Fixed Aggregation Trade-off

In autonomous driving, LiDAR sensors provide a sparse representation of the environment. Aggregating multiple frames helps fill in geometric details for stationary objects. However, for fast-moving vehicles, excessive aggregation leads to "smearing," where the point cloud becomes distorted and misaligned with the object's true bounding box.

Performance trade-off: Stationary objects benefit from more frames, while fast-moving objects suffer performance degradation due to motion blur.

As shown in the performance analysis, stationary objects see a steady increase in Average Precision (AP) as more frames are added. In contrast, fast-moving objects peak early and then experience a sharp decline in accuracy as motion-induced noise overwhelms the geometric benefits of additional points.

Methodology

VADet moves away from the "one-size-fits-all" approach through a two-stage strategy:

  1. Random Aggregation Training (RAT): To avoid the cost of training separate models for every possible frame count, VADet utilizes RAT. A single detector is trained where the number of aggregated frames varies randomly for each scene. This ensures the model is robust to varying input densities and allows for fair evaluation across different frame counts.
  2. Variable Aggregation Function ($\eta$): The core of the system is a lookup table that maps an object's estimated speed and point density to an empirically optimal number of frames.
  3. Per-Object Input Construction: VADet uses a constant velocity model to predict an object's prior locations. It then crops and aggregates the specific number of frames required for that object's region, providing a "customized" point cloud to the detector.

Key Results and Evaluation

The researchers evaluated VADet on the Waymo Open Dataset. By integrating VADet with the VoxelNeXt architecture, the team achieved 76.6 L2 AP on the validation set and 79.8 L2 AP on the test set, outperforming complex two-stage and multi-modal detectors.

Qualitative comparison: (a) 3-frame aggregation is sparse; (b) 16-frame aggregation causes smearing for fast vehicles; (c) VADet adaptively balances density and clarity.

Qualitative results highlight VADet's ability to maintain clear object boundaries. While 16-frame aggregation creates significant "tails" behind moving vehicles, VADet's adaptive approach provides enough density for detection without the distorting effects of motion blur. Furthermore, VADet is highly efficient, adding only ~50ms of overhead, making it far more practical for real-time applications than high-latency attention-based fusion methods.

Conclusion

VADet demonstrates that optimizing the input data through adaptive aggregation is a powerful alternative to increasing model complexity. By accounting for both speed and point density, VADet provides a consistent performance boost across various base architectures (CenterPoint, VoxelNeXt, DSVT). Future work may involve replacing the empirical lookup table with a learned function to map features directly to optimal aggregation strategies.

Summary updated

Abstract

Input aggregation is a simple technique used by state-of-the-art LiDAR 3D object detectors to improve detection. However, increasing aggregation is known to have diminishing returns and even performance degradation, due to objects responding differently to the number of aggregated frames. To address this limitation, we propose an efficient adaptive method, which we call Variable Aggregation Detection (VADet). Instead of aggregating the entire scene using a fixed number of frames, VADet performs aggregation per object, with the number of frames determined by an object's observed properties, such as speed and point density. VADet thus reduces the inherent trade-offs of fixed aggregation and is not architecture specific. To demonstrate its benefits, we apply VADet to three popular single-stage detectors and achieve state-of-the-art performance on the Waymo dataset.
BibTeX
@inproceedings{huang2025vadet,
  author = {Chengjie Huang and Vahdat Abdelzad and Sean Sedwards and Krzysztof Czarnecki},
  title = {{VADet: Multi-Frame LiDAR 3D Object Detection Using Variable Aggregation}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2025},
  doi = {10.1109/wacv61041.2025.00079},
}
Figure