Abstract
Material appearance is a key component of photorealism, with a pronounced impact on human perception. Although there are many prior works targeting at measuring opaque materials using light-weight setups (e.g., consumer-level cameras), little attention is paid on acquiring the optical properties of translucent materials which are also quite common in nature. In this paper, we present a practical method for acquiring scattering properties of translucent materials, based solely on ordinary images captured with unknown lighting and camera parameters. The key to our method is an inter-pixel translucency prior which states that image pixels of a given homogeneous translucent material typically form curves (dubbed translucent curves) in the RGB space, of which the shapes are determined by the parameters of the material. We leverage this prior in a specially-designed convolutional neural network comprising multiple encoders, a translucency-aware feature fusion module and a cascaded decoder. We demonstrate, through both visual comparisons and quantitative evaluations, that high accuracy can be achieved on a wide range of real-world translucent materials.
Introduction
Capturing the appearance of translucent materials—such as wax, soap, or plastic—is a significant challenge in computer vision. Unlike opaque objects, where light reflects primarily off the surface, light enters translucent objects, undergoes subsurface scattering, and exits at different points. Modeling this requires complex parameters: the extinction coefficient (light travel distance), scattering albedo (scattering vs. absorption), and the phase function (scattering direction).
Traditionally, measuring these properties required "lab-grade" setups involving darkrooms, lasers, and structured lighting. While deep learning has simplified opaque material measurement, it often fails for translucency because it ignores the non-local nature of internal light transport. This paper introduces a practical approach using only a few smartphone photos under unknown lighting conditions.
The Inter-Pixel Translucency Prior
The core innovation is the discovery of the "inter-pixel translucency prior." When the RGB values of pixels from a homogeneous translucent material are plotted across varying lighting conditions, they form distinct "translucent curves" in RGB space. The specific shape and trajectory of these curves serve as a unique fingerprint for the material's optical properties.
By analyzing how pixel values change as a light source (like a phone flash) moves, the method extracts structural information about the material. To assist the learning process, the authors derive two "pseudo-maps":
- Pseudo-albedo map (A): A proxy for the material's base color.
- Bias map (B): A map that highlights "translucent edges"—areas where light bleeds through thin geometry—which are crucial for determining how "thick" or "dense" the material is.
IPTNet Architecture
The proposed IPTNet architecture processes raw images alongside these pseudo-maps through a multi-branch CNN. A "translucency-aware" feature fusion module uses the bias map to tell the network which pixels contain the most information about scattering.
The network employs a cascaded decoder to decouple parameter estimation. Instead of predicting all parameters at once, the network first predicts the albedo, then uses that information to refine its prediction of the extinction coefficient and phase function. This approach respects the "principle of similarity" in physics, where different parameter combinations can appear identical, by using reduced, less correlated parameters ($\sigma'_t, \Lambda'$).
Experimental Results
IPTNet was trained on a massive synthetic dataset of 16,800 image groups rendered with the Mitsuba 3 engine. It was then evaluated against state-of-the-art methods on both synthetic benchmarks and real-world objects.
Key findings include:
- High Accuracy: IPTNet significantly outperformed previous state-of-the-art methods (like Inverse Transport Networks) across all metrics (MSE, PSNR, and SSIM). It was particularly better at recovering the "chromaticity" (color accuracy) of the materials.
- Efficiency: Compared to differentiable rendering techniques, IPTNet is lightning fast. It takes 30 milliseconds to extract material properties, whereas optimization-based methods can take 20 minutes per scene.
- Real-World Robustness: The method successfully reconstructed real-world objects like orange juice at different concentrations and waxy blocks of varying thicknesses, matching the visual "feel" of the reference photos almost perfectly.
Conclusion
While highly effective, the method currently requires at least three images and can struggle with near-transparent materials (like clear glass) where background interference breaks the translucency prior. Specular highlights on very shiny surfaces can also introduce noise. Future research aims to extend this framework to inhomogeneous materials, such as veined marble or fruit, where scattering properties vary internally.
Summary updated