Distribution Aware VoteNet for 3D Object Detection

Junxiong Liang, Pei An, Jie MaAAAI 2022 · DOI

Introduction

3D object detection in indoor scenes is a fundamental task for robotics and augmented reality. However, real-world point clouds are often sparse, noisy, and incomplete due to occlusions and sensor limitations. DAVNet (Distribution Aware VoteNet) addresses these challenges by modeling bounding box parameters as probability distributions rather than deterministic values.

What Problem Does This Paper Address?

In cluttered 3D environments, objects are rarely captured in their entirety. For instance, a chair might be partially obscured by a desk, or a sofa might be truncated by the camera's field of view.

Challenges in 3D object detection: (a) boundary uncertainty due to occlusion and (b) label ambiguity for similar objects.

Current 3D detectors, such as the popular VoteNet, typically employ point estimates (mathematically known as Dirac delta distributions) for bounding box regression. This approach fails to account for the inherent uncertainty of occluded boundaries. If only the top of an object is visible, its bottom boundary is ambiguous. Forcing a model to predict a single "ground truth" coordinate leads to unstable training and poor localization. Additionally, standard methods use fixed-size regions for feature extraction, which struggle to adapt to the varying scales of indoor objects.

Methods

DAVNet introduces three key architectural innovations to improve 3D object detection:

  1. Distribution-Aware Regression (DAR): Instead of regressing a single offset for the box center or size, DAVNet predicts a probability distribution over a range of possible values. The final prediction is derived by integrating over this distribution, allowing the model to represent spatial uncertainty explicitly.
  2. Scale Adaptive (SA) Encoder: To handle objects of different sizes, the SA encoder generates a 4x4x4 grid within an initial box proposal. It then interpolates features based on this grid, effectively resizing the receptive field to match the object's dimensions.
  3. Joint Localization Quality Estimator (JLQE): This module predicts a confidence score by combining physical features with the statistics of the predicted distribution. A "sharp" distribution (low variance) indicates high localization certainty, which is used to refine the final detection scores and suppress false positives.

Datasets

The model was rigorously evaluated on two major indoor 3D benchmarks:

  • ScanNet V2: A large-scale dataset containing 1,513 reconstructed 3D indoor scenes.
  • SUN RGB-D: A dataset of approximately 10,000 single-view RGB-D snapshots, characterized by significant occlusion and incomplete object geometry.
A typical indoor scene from the datasets, showcasing the complexity and potential for occlusion in real-world environments.

Key Insights

  • Sharpness as a Proxy for Quality: The authors observed a strong correlation between the "sharpness" of the predicted distribution and the Intersection over Union (IoU) score. This allows the model to identify when it has found a clear boundary versus when it is guessing due to ambiguity.
  • Adaptive Context: By moving away from fixed-radius grouping, the SA encoder ensures that the model captures the appropriate amount of context for both small items (like lamps) and large furniture (like sofas).

Key Results

DAVNet achieves state-of-the-art performance on both benchmarks:

  • SUN RGB-D: Reached a mean Average Precision ([email protected]) of 60.3% and a significant [email protected] of 39.4% (a substantial jump from VoteNet's 31.7%).
  • ScanNet V2: Achieved an [email protected] of 50.2%, outperforming previous methods like MLCVNet (41.4%).

Conclusion and Future Work

DAVNet demonstrates that modeling uncertainty through probability distributions significantly enhances 3D object localization. While the model shows some sensitivity to data scarcity for rare categories, its ability to handle occlusion and scale variation sets a new standard for indoor 3D detection. Future work may explore multi-modal distributions to handle even more complex spatial ambiguities where an object might have multiple likely locations.

Summary updated

BibTeX
@inproceedings{liang2022distribution,
  author = {Junxiong Liang and Pei An and Jie Ma},
  title = {{Distribution Aware VoteNet for 3D Object Detection}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2022},
  doi = {10.1609/aaai.v36i2.20049},
}
Figure