Volumetric reconstruction applied to perceptual studies of size and weight

Jonathan Balzer, Megan A. K. Peters, Stefano Soatto

TL;DR

This paper introduces "Yet Another Scanner" (YAS), a novel structured-light scanning system designed for accurate volumetric reconstruction of small-to-medium sized objects. YAS aims to provide reliable ground truth data for cognitive neuroscience studies, particularly for investigating the size-weight illusion, by overcoming systematic biases found in commercial scanners.

What Problem Does This Paper Address?

Accurately measuring the physical properties of objects, like volume, is crucial for understanding human perception, especially in cross-modal integration (e.g., vision and touch). The "size-weight illusion," where smaller objects are perceived as heavier, highlights how our senses can be systematically biased. To quantitatively study such illusions, researchers need precise 3D models of objects. However, existing commercial structured-light scanning systems often suffer from systematic biases and inaccuracies that make them unsuitable for rigorous scientific investigation. This paper addresses the need for a reliable, accessible, and accurate volumetric reconstruction system for scientific applications.

Methods

The authors developed "Yet Another Scanner" (YAS), a structured-light scanning system with two key innovations:

  • Sparse Viewpoint Sampling: Unlike dense sampling methods (like Kinect Fusion), YAS uses a sparse set of viewpoints. This reduces data redundancy and the impact of tracking errors, leading to more robust reconstructions.
  • Screened Poisson Reconstruction: Instead of directly building a distance field, YAS recovers the distance function from its gradient using a screened Poisson reconstruction. This method is highly resilient to noise and preserves high-frequency signal details, crucial for accurate surface and volume estimation.

The system also incorporates a robust wide-baseline matching approach for aligning depth maps, which is less prone to drift than tracking-based methods. For volume estimation, they leverage a method based on the Gauss divergence theorem, adapted for surfaces with boundaries (like objects resting on a plane).

Datasets

The paper evaluates their system on a dataset of 20 specimens. This dataset is divided into two groups:

  • Geometric Primitives: Cubes and cylinders, whose dimensions can be measured manually with a ruler.
  • Free-Form Objects: Objects with more complex shapes, ranging from a few centimeters to the size of an upper body.

The "ground truth" volumes for the geometric primitives were determined by manual measurement, acknowledging that these measurements have their own inherent uncertainty.

Key Insights and Ideas

  • Commercial Scanners Have Systematic Biases: The study reveals that even visually pleasing reconstructions from commercial systems like Kinect Fusion can have significant systematic errors, particularly overestimating volume. This highlights the critical need for quantitative analysis beyond visual inspection for scientific applications.
  • Calibration is Paramount: The authors emphasize that a good calibration can significantly improve reconstruction accuracy, sometimes by an order of magnitude. This is a crucial takeaway for anyone using depth sensors for measurement.
  • Sparse Sampling Can Be Advantageous: Contrary to the trend of dense data acquisition, YAS's sparse sampling approach, combined with robust alignment, proves effective and reduces issues like stratification and drift.
  • Screened Poisson Reconstruction is Robust: The choice of screened Poisson reconstruction is shown to be resilient to noise and capable of preserving important details, making it suitable for generating accurate volumetric models.

Key Results

  • YAS Performance: YAS achieved an average relative volume error of -0.34% for geometric primitives, outperforming both Scenect (-5.74%) and Kinect Fusion (24.2%) in terms of accuracy.
  • Kinect Fusion Overestimation: Kinect Fusion systematically overestimated the ground truth volume by a significant margin (average error of 24.2%), which the authors attribute to calibration issues.
  • Scenect Performance: Scenect showed comparable performance to YAS on geometric primitives but performed worse on free-form objects.
  • Topological Quality: While Kinect Fusion produced visually appealing models, their topological quality was inferior, containing non-manifold simplices.

Limitations and Future Work

  • No Real-Time Capability: YAS is not a real-time system, which is acceptable for their target application of offline scanning but might be a limitation for other uses.
  • Texture Dependence: The correspondence matching relies on photometry, meaning the approach may struggle with objects lacking sufficient texture. However, the authors note that background texture can sometimes compensate.
  • Future Work: The authors suggest that the bottleneck in their system is the matching process, which could be improved with approximate nearest-neighbor search. They also plan to integrate Poisson reconstruction directly into their code and make their code and data publicly available.

Summary updated

Abstract

We explore the application of volumetric reconstruction from structured-light sensors in cognitive neuroscience, specifically in the quantification of the size-weight illusion, whereby humans tend to systematically perceive smaller objects as heavier. We investigate the performance of two commercial structured-light scanning systems in comparison to one we developed specifically for this application. Our method has two main distinct features: First, it only samples a sparse series of viewpoints, unlike other systems such as the Kinect Fusion. Second, instead of building a distance field for the purpose of points-to-surface conversion directly, we pursue a first-order approach: the distance function is recovered from its gradient by a screened Poisson reconstruction, which is very resilient to noise and yet preserves high-frequency signal components. Our experiments show that the quality of metric reconstruction from structured light sensors is subject to systematic biases, and highlights the factors that influence it. Our main performance index rates estimates of volume (a proxy of size), for which we review a well-known formula applicable to incomplete meshes. Our code and data will be made publicly available upon completion of the anonymous review process.
BibTeX
@inproceedings{balzer2014volumetric,
  author = {Jonathan Balzer and Megan A. K. Peters and Stefano Soatto},
  title = {{Volumetric reconstruction applied to perceptual studies of size and weight}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2014},
  url = {https://openreview.net/forum?id=8tjkucZ4GA},
}
Figure