Shape From Caustics: Reconstruction of 3D-Printed Glass From Simulated Caustic Images

Marc Kassubeck, Florian Burgel, Susana Castillo, Sebastian Stiller, Marcus MagnorWACV 2021

TL;DR

This paper introduces a novel computational framework for reconstructing the 3D shape of glass objects from their caustic images, specifically targeting the needs of 3D glass printing. By combining differentiable light propagation simulation with a regularization scheme that incorporates manufacturing process knowledge, they enable accurate shape inference from indirect visual cues.

What Problem Does This Paper Address?

Additive manufacturing of glass, particularly techniques like laser-based fused glass deposition (LGD), offers unprecedented control over complex optical element fabrication. However, a significant challenge is reliably controlling this process. Current methods for verifying the printed 3D shape are often indirect and time-consuming, relying on external measurement equipment. This lack of real-time, in-situ feedback hinders efficient error correction or rejection of faulty workpieces. The authors aim to bridge this gap by using caustics – the patterns of light focused or reflected by a transparent object – as a rich source of information to infer the glass object's shape directly within the manufacturing environment.

Methods

The core of the proposed solution lies in an "inverse rendering" approach. They developed a differentiable forward simulation operator that models how light propagates through a 3D glass object and forms caustic patterns on a receiver. This operator is built upon a photon mapping algorithm adapted for GPU acceleration and uses a custom CUDA kernel for efficient computation.

For reconstruction, they tackle the ill-posed nature of inferring shape from caustics by employing a regularization scheme. This scheme incorporates several pieces of prior knowledge:

  • Sparsity: The assumption that the glass object's shape can be represented efficiently in either a pixel or wavelet basis.
  • Physical Bounds: Constraints based on the known physical limits of the 3D printer (e.g., the material is deposited within a specific area).
  • Volume Conservation: The knowledge of the total volume of glass used in the printing process.

They explore different reconstruction algorithms, including pixel-based and wavelet-based Landweber schemes, and introduce novel penalty terms to enforce these priors. A key innovation is the integration of a volume heuristic that helps to accelerate convergence and improve accuracy.

Datasets

The paper primarily uses simulated data generated by their own forward caustic renderer. They also present results on a real-world sample, photographed under controlled lighting conditions. No new benchmark datasets are introduced.

Key Insights and Ideas

  • Caustics as a Rich Shape Cue: The paper demonstrates that caustic patterns, often seen as a visual artifact, contain significant information about the underlying 3D shape of transparent objects.
  • Differentiable Simulation for Inverse Problems: By making the light transport simulation differentiable, they can efficiently compute gradients needed for optimization-based shape reconstruction.
  • Leveraging Manufacturing Priors: Incorporating knowledge about the 3D printing process (deposition area, volume) is crucial for achieving accurate and physically plausible reconstructions from noisy and indirect measurements.
  • Multi-faceted Regularization: Combining sparsity, physical bounds, and volume conservation provides a robust framework for solving the ill-posed inverse problem.
  • Practicality for In-Situ Monitoring: The proposed method is designed with the goal of integration into manufacturing workflows, enabling real-time feedback.

Key Results

  • The proposed method successfully reconstructs 3D glass shapes from simulated caustic images, outperforming naive gradient descent approaches.
  • Experiments show that incorporating volume heuristics (V1) significantly improves height field error (from 11% to 7%) and reduces runtime.
  • Comparisons with a caustic design approach (Schwartzburg et al.) show that their regularization prevents implausible shapes and achieves lower surface error.
  • The method demonstrates robustness to noise and can handle scenes with multiple refractive elements, a capability not commonly found in prior work.
  • A real-world sample reconstruction shows promising accuracy for feedback loops, though fine details and precise dimensions remain challenging.

Limitations and Future Work

  • Reconstruction of Small Features: The paper acknowledges that reconstructing very thin print lines remains challenging due to the strong curvature and limited receiver resolution.
  • Material Properties: The current method assumes a uniform refractive index. Extending it to handle complex absorption coefficients and frequency-dependent refractive indices for a wider range of materials is suggested.
  • In-Situ Measurement Setup: While the method is designed for in-situ use, achieving the required accuracy for industrial applications may necessitate improvements in screen materials and light sources.
  • Multi-Angle and Multi-Wavelength: While the framework supports multi-wavelength simulation, the reconstruction is primarily demonstrated for single wavelengths. Further exploration of multi-angle and multi-wavelength data could improve reconstruction quality.

Summary updated

BibTeX
@inproceedings{kassubeck2021shape,
  author = {Marc Kassubeck and Florian Burgel and Susana Castillo and Sebastian Stiller and Marcus Magnor},
  title = {{Shape From Caustics: Reconstruction of 3D-Printed Glass From Simulated Caustic Images}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2021},
}
Figure