Consistency and confidence: A dual metric for verifying 3D object detections in multiple LiDAR scans

David Doria, Richard J. RadkeWorkshops @ ICCV 2009 · DOI

Introduction

Detecting 3D objects in LiDAR point clouds typically involves aligning a known mesh to partial, noisy data. While Iterative Closest Point (ICP) is the standard for alignment, its output score is often inadequate for verification. ICP scores are frequently ambiguous, scale-dependent, and "physically blind"—they do not distinguish between a model placed in empty space and one placed behind an obstruction. This paper proposes a dual-metric framework—consistency and confidence—to provide a physically meaningful, unitless verification method that requires no training data.

The Consistency Measure

The consistency metric serves as a "physical possibility" check. It is based on the principle of free space violation: if a LiDAR ray reaches a background point, the space along that ray must be empty. If a hypothesized model intersects this path, the hypothesis is physically inconsistent.

Geometric representation of the consistency check: rays from the scanner intersect the model surface and scene points to determine physical plausibility.

As illustrated in Figure 2, for every ray intersecting the model at distance $d_m$ and the sensed point at $d_s$, the system evaluates consistency. If $d_m < d_s$, the model is in front of the sensed point, which is physically consistent (representing an occlusion). However, if $d_m > d_s$, the ray would have had to pass through the model to reach the sensed point, marking the placement as inconsistent. This asymmetry is crucial for distinguishing between valid occlusions and impossible placements.

The Confidence Measure

While consistency ensures a model could be there, confidence measures the evidence that it is there. A model hidden behind a wall is consistent but lacks evidence. To quantify this, each vertex of the 3D model is assigned an information value. As LiDAR points are detected near these vertices, the "observed information" is updated using a Gaussian rule. This allows the metric to scale from 0 to 1, where 1 represents a fully observed model. This metric is particularly useful for signaling when a match is likely correct but based on limited data due to heavy occlusion.

Experimental Evaluation

The framework was tested using a high-precision cat sculpture, synthetic vehicle models, and real-world parking lot data. These experiments demonstrated the system's ability to disambiguate similar shapes across multiple viewpoints.

Visualizing the impact of model placement on verification metrics: (a) correct alignment, (b) model placed behind sensed points (inconsistent), and (c) model placed in front (occluded but consistent).

In the cat sculpture experiment (Figure 4), the metrics successfully distinguished between correct and incorrect placements. Notably, placing the model behind the scene (b) resulted in low consistency, while placing it in front (c) maintained consistency but lowered confidence. In real-world parking lot tests, the consistency metric showed a significantly larger gap between correct (0.589) and incorrect (0.077) matches compared to standard ICP costs, which were nearly indistinguishable (0.057 vs 0.094).

Conclusion

The dual-metric approach offers a robust, interpretable alternative to traditional registration scores. By leveraging physical constraints and multi-view data, it effectively handles occlusions and disambiguates similar object shapes. Future work will focus on improving computational efficiency through GPU acceleration and exploring non-model-based verification strategies that do not rely on pre-existing CAD meshes.

Summary updated

BibTeX
@inproceedings{doria2009consistency,
  author = {David Doria and Richard J. Radke},
  title = {{Consistency and confidence: A dual metric for verifying 3D object detections in multiple LiDAR scans}},
  booktitle = {Workshops @ ICCV},
  year = {2009},
  doi = {10.1109/iccvw.2009.5457438},
}
Figure