Polarization as Texture: Microscale 3D Shape from Polarized Light Focus

Ren Matsumoto, Takahiro Okabe, Ryo KawaharaWACV 2025

TL;DR

This paper introduces a novel method for reconstructing the 3D shape of microscale objects by treating polarized light as a texture cue for Shape-from-Focus (SfF). By leveraging polarization's sensitivity to surface orientation, the method overcomes the limitations of traditional SfF techniques that struggle with textureless surfaces, enabling accurate depth estimation for a wider range of micro-objects.

What Problem Does This Paper Address?

Reconstructing the 3D shape of microscopic objects is crucial for fields like medicine and material science. A common challenge in this domain is the shallow depth of field in microscopy, which leads to blurred images. Shape-from-Focus (SfF) is a passive technique that uses this blur to infer depth, but it heavily relies on surface texture to measure focus levels. This makes SfF ineffective for textureless objects. While polarization of light can reveal surface orientation, existing Shape-from-Polarization (SfP) methods are often complex, sensitive to viewing angles, and can also be affected by blur. This paper addresses the gap of accurately reconstructing microscale 3D shapes, especially for textureless objects, by combining the strengths of SfF and polarization.

Methods

The core innovation lies in treating polarized light as a rich texture source for SfF. The authors propose:

  • Polarimetric Focus Measure: Instead of relying on traditional image gradients, they use the Stokes vector components (representing polarization information) to calculate focus measures. This is based on a derived blurring model for polarization images, treating polarization as a texture that changes with focus.
  • Polarization-Guided Belief Propagation: To improve the accuracy and robustness of depth estimation, they employ a belief propagation (BP) algorithm. This algorithm uses polarization-derived normal cues to create a prior distribution of focus levels between neighboring pixels. This "iso-depth contour" concept helps to propagate focus information smoothly and accurately across the image, even in areas with weak local focus cues.
  • Coaxial Falloff Illumination: A specific lighting setup is used where the light source is aligned with the camera's viewing direction. This simplifies the polarization model by primarily capturing diffuse reflection, making the polarization signal more directly related to surface normals.

Datasets

The paper evaluates its method using:

  • Synthetic Data: Generated using the pBRDF dataset, simulating microscale objects (Sphere, Bunny, Cheburashka) with varying surface irregularities and adding Gaussian noise to mimic real-world conditions.
  • Real-world Objects: Tiny objects such as a Sphere, Chrysanthemum seed, and Pinecone were used for qualitative evaluation.

No new datasets were introduced.

Key Insights and Ideas

  • Polarization as Texture: The fundamental insight is that polarization patterns, arising from surface orientation, can serve as a robust "texture" for focus-based depth estimation, even on otherwise textureless surfaces.
  • Stokes Vector for Focus: Using the Stokes vector components (S0, S1, S2) provides richer information for focus measurement than intensity alone, allowing for better discrimination of in-focus regions.
  • Belief Propagation for Global Consistency: Applying belief propagation guided by polarization normals significantly improves the global consistency of depth maps, overcoming local ambiguities and noise in focus measures.
  • Overcoming SfF and SfP Limitations: The method effectively bridges the gap between SfF (texture-dependent) and SfP (blur-sensitive), offering a more versatile solution for microscale 3D reconstruction.

Key Results

  • Quantitative Accuracy: The method achieved state-of-the-art results on synthetic data, outperforming existing SfF and SfP methods in terms of depth reconstruction accuracy (RMSE) and normal estimation (MAE), especially on textureless surfaces. For example, on the Sphere object, the RMSE was 0.027 mm, and the MAE was 6.42 degrees.
  • Robustness to Irregularity: The method demonstrated robustness to varying levels of surface irregularity, unlike some SfP methods that assume smooth surfaces.
  • Qualitative Performance: Real-world experiments on objects like spheres, seeds, and pinecones showed visually accurate 3D point clouds and normal maps, confirming the method's effectiveness.
  • Ablation Study: An ablation study confirmed the effectiveness of both the polarimetric focus measure and the polarization-guided belief propagation, showing significant improvements over simpler approaches.

Limitations and Future Work

  • Limited Focal Stacks: As an SfF method, depth accuracy can be compromised if only a few focal stacks are available.
  • Diffuse Reflection Reliance: The method primarily relies on the polarization of diffuse reflections from opaque objects. It may not perform well on perfectly transparent objects or those with strong inter-reflections.
  • Black Objects: The method struggles with entirely black objects.
  • Future Work: The authors suggest exploring the use of specular polarization components, analyzing surface properties like refraction index and roughness, and extending the method to handle more complex optical phenomena.

Summary updated

BibTeX
@inproceedings{matsumoto2025polarization,
  author = {Ren Matsumoto and Takahiro Okabe and Ryo Kawahara},
  title = {{Polarization as Texture: Microscale 3D Shape from Polarized Light Focus}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2025},
}
Figure