Polarimetric Disentanglement for Inverse Rendering

This direction is split into other directions: specular-diffuse separation. Its last version stays readable and citable here.

Version 1 of 1174 papers277 conceptsfrozen 2026-10-04

In standard computer vision, we treat an image as a grid of intensity values. However, a single pixel’s brightness is actually a "blended" signal—a sum of light reflected off a surface (specular), light scattered from within a material (diffuse), and sometimes light reflected from a foreground pane of glass.

Traditional algorithms struggle to "unmix" these components because the problem is mathematically underdetermined. Polarimetric inverse rendering solves this by exploiting the hidden physical state of light: its polarization. By measuring how light waves oscillate, researchers can treat the scene not as a flat image, but as a collection of distinct physical layers that can be mathematically separated.

The Physics of Unmixing

The core intuition behind this research direction is that different types of light-matter interaction leave unique "fingerprints" on the polarization state of light.

When light hits a non-metallic surface, it generally splits into two paths. Some light reflects immediately off the surface; this is the specular reflection (the "shine"). Because this light barely interacts with the material's molecules, it largely maintains the polarization state of the source. The rest of the light enters the material, bounces around (subsurface scattering), and then exits. This is the diffuse reflection (the "color"). This internal bouncing randomizes the light's orientation, effectively depolarizing it.

As Olivier Maury demonstrates in High-Res Facial Appearance Capture from Polarized Smartphone Images, we can use this difference to physically "strip" a face of its shine. By placing a polarizing filter on a light source and another on a camera, we can capture a "cross-polarized" image that blocks the specular shine entirely, leaving only the pure skin pigment. This physical decoupling prevents "baked-in" highlights, allowing digital models to be realistically relit in any environment.

Resolving Geometric Ambiguity

Polarization doesn't just separate colors; it reveals shape. The way light polarizes upon reflection is tied to the angle of the surface it hit. While a standard image might see a "flat" white wall, a polarimetric sensor sees a gradient of polarization states that correspond to the wall's subtle curvature.

However, polarization data is notoriously ambiguous—a single polarization state could represent two different surface orientations (the "azimuth ambiguity"). Modern research resolves this by fusing polarization with other cues. Achuta Kadambi and Vage Taamazyan showed in Shape from Mixed Polarization that by modeling both diffuse and specular polarization simultaneously, one can recover accurate 3D normals even for complex, real-world objects that are neither perfectly matte nor perfectly shiny.

Even natural light can be used as a tool. In Shape From Sky, Tomoki Ichikawa and Ko Nishino treat the entire sky as a polarized light source. By modeling how the sky's polarization pattern reflects off an object, they can recover fine geometric details from a single image, effectively turning the atmosphere into a giant laboratory instrument.

Integrating Physics into Neural Fields

The most recent frontier involves embedding these physical laws into neural representations like NeRF (Neural Radiance Fields) and 3D Gaussian Splatting.

Standard neural rendering often "hallucinates" fake geometry to explain away moving highlights—creating "floaters" or blurred surfaces. Akshat Dave and Ashok Veeraraghavan addressed this with PANDORA, which uses Stokes vectors to represent light. By forcing the neural network to obey the rules of polarization, the system learns to distinguish between a white spot that is part of the object's texture and a white spot that is a view-dependent reflection.

Similarly, Bo Guo and Jia Li introduced PolarGS, which uses polarimetric cues to supervise the optimization of 3D Gaussian Splats. By isolating the diffuse component, the system ensures that the underlying 3D geometry remains stable even as the camera moves through complex lighting.

Seeing Through the Noise

Beyond 3D reconstruction, this "unmixing" capability is a superpower for seeing through scattering media like fog, water, or glass.

In Learning to dehaze with polarization, Chao Xu and Chu Zhou demonstrate that haze and "airlight" have distinct polarization signatures. By modeling the polarization of both the background light and the target object, they can mathematically "subtract" the fog from the image.

This same principle applies to removing glass reflections. As Chenyang Lei shows in Polarized Reflection Removal With Perfect Alignment in the Wild, because reflected light is highly polarized compared to transmitted light, we can use the Angle of Polarization (AoP) as a reliable mask to separate what is behind the glass from what is reflected on it.

Whether it is detecting a road in total darkness using infrared polarization (Ning Li) or identifying a spoofing attack on a face scanner (PARAPH), the core idea remains the same: polarization provides the physical constraints necessary to solve the "unmixing" problems that intensity alone cannot.

Papers in this version

Showing 60 of 174 papers.

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_polarimetric_disentanglement_for_inverse_b74093e2,
  title = {Polarimetric Disentanglement for Inverse Rendering},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/polarimetric-disentanglement-for-inverse-rendering/rdr_b74093e2ae684d4b9959f4a4e3685757},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_polarimetric_disentanglement_for_inverse_b74093e2_v1,
  title = {Polarimetric Disentanglement for Inverse Rendering},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/polarimetric-disentanglement-for-inverse-rendering/rdr_b74093e2ae684d4b9959f4a4e3685757/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}