TL;DR
This paper highlights a critical flaw in 3D object detection research: new architectures are often compared to older baselines without controlling for computational cost (latency). By simply scaling up SECOND, a classic and "simple" one-stage detector, the authors show it can match or outperform modern, highly complex state-of-the-art models like PV-RCNN++ when given the same "latency budget."
What Problem Does This Paper Address?
In the world of 2D image recognition (like ImageNet), it is standard practice to compare models within the same "compute regime" (e.g., comparing models with similar FLOPs or parameters). However, the authors observe that in LiDAR-based 3D object detection, this discipline is largely missing.
Recent papers often claim superior performance based on novel, complex architectural components (like sophisticated point-voxel fusion or multi-stage refinement). But because these new models are often much slower than the baselines they compare against, it’s unclear if their gains come from "smarter" design or simply from using more "horsepower." This paper aims to level the playing field through cost-aware evaluation.
Methods
The authors take SECOND, a well-known, grid-based one-stage detector, and systematically scale it up to see how much performance is left on the table. They explore three primary scaling dimensions:
- Network Depth: Increasing the number of layers.
- Network Width: Increasing the number of channels.
- Pre-head Resolution: Increasing the spatial dimension of the feature map right before it enters the detection head.
Key Technical Innovations:
* Pre-head Resolution Scaling: They find that reducing downsampling in the 3D backbone (keeping a higher resolution feature map) is more effective than just adding depth or width. This allows for a denser sampling of object anchors.
* Residual Connections: They note that simply deepening 3D sparse convolutional networks leads to optimization issues. Adding residual connections (res versions) is essential to unlock the potential of deeper backbones.
* Head Comparison: They evaluate both traditional Anchor-based heads and more recent Center-based heads (from CenterPoint) across different scales.
Datasets
The study exclusively uses the Waymo Open Dataset (WOD), currently the largest and most rigorous public benchmark for LiDAR-based 3D object detection. They utilize both the full training set and a "20% mini-set" for efficient ablation studies.
Key Insights and Ideas
- The "Simple" Baseline is a Giant: A properly scaled SECOND model (dubbed
A1resorA2res) easily outperforms many complex methods published in the last year. It can even match the performance of the current state-of-the-art, PV-RCNN++, if both are allowed the same inference latency. - Resolution is King: In 3D detection, the size of an object doesn't change with distance (unlike 2D images). Therefore, the primary benefit of higher resolution isn't "seeing small things" better, but rather providing a denser grid for anchors and keypoints to land on, which significantly boosts localization accuracy.
- Diminishing Returns of Complexity: The performance gap between simple one-stage detectors and complex two-stage detectors shrinks significantly as the backbone size increases.
- The "Center Head" Advantage Flips: While Center-based heads are much better than Anchor-based heads on small backbones, the paper finds that on very large backbones, the Anchor-based head can actually catch up or even slightly overtake the Center head in certain metrics.
Key Results
- Efficiency:
SECOND-Anchor-A1resachieves a similar Mean Average Precision weighted by Heading (mAPH) toPV-RCNN++ (Anchor)but is 2x faster (67ms vs. ~130ms+). - Performance: Scaled SECOND models outperformed recent specialized methods like
VoTr-TSDandPyramid-PVin both speed and accuracy. - Scaling Strategy: The authors demonstrate that "compound scaling" (increasing depth, width, and resolution simultaneously) is generally superior to scaling a single dimension, echoing findings from 2D vision (like EfficientNet).
Limitations and Future Work
- Scope of Baselines: The paper focuses almost entirely on scaling the SECOND architecture. While SECOND is a foundational component of many other models, it remains to be seen if these scaling laws apply identically to "range-view" or purely "point-based" detectors.
- Hardware Specificity: Latency was measured on a specific setup (RTX 3090 GPU). Since sparse convolutions perform differently across hardware generations, the "cost" might shift on edge devices or different GPU architectures.
- Call to Action: The authors strongly recommend that the 3D detection community adopt cost-aware benchmarks and include "Scaled SECOND" as a standard baseline to ensure that future "architectural innovations" are truly providing value beyond just increased computation.
Summary updated