Polarization Guided Mask-Free Shadow Removal

Chu Zhou, Chao Xu, Boxin ShiAAAI 2025 · DOI

TL;DR

This paper introduces Pol-ShaRe, the first method to remove shadows from images using polarization information, eliminating the need for manual shadow masks. By leveraging both intensity and polarization cues, Pol-ShaRe effectively decouples texture preservation from brightness/color restoration, leading to fewer artifacts and state-of-the-art performance on synthetic and real-world images.

What Problem Does This Paper Address?

Shadows are a common visual artifact that degrades image quality and hinders the performance of downstream computer vision tasks. While existing shadow removal techniques have made progress, they often fall into two categories: those requiring an externally generated shadow mask (which can be inaccurate and labor-intensive to obtain) or those that attempt to remove shadows without a mask but struggle with artifacts like false colors and blurred textures due to the inherent ill-posedness of the problem. This paper aims to address these limitations by proposing a novel, mask-free shadow removal approach that utilizes polarization information to guide the process.

Methods

The core of Pol-ShaRe is a two-stage pipeline designed to tackle the ill-posed nature of shadow removal.

  • Dual Domain Priors: The method leverages two types of priors:
  • Intensity Domain Prior (Pint): Derived from the observation that shadowed regions typically have lower pixel values and a bluer tint due to skylight.
  • Polarization Domain Prior (Ppol): Based on the degree of polarization (DoP), which is often higher in shadowed regions and can provide distinct texture information.
  • Two-Stage Pipeline:
    • Stage 1: Focuses on reconstructing a "modulated intensity" (m), which is less affected by shadow and preserves texture/structure information. This stage uses the dual domain priors to guide the process.
  • Stage 2: Reconstructs the final shadow-free image (I) using the original image (I*), the intensity domain prior (Pint), and the restored modulated intensity (m) as guidance. This decouples brightness and color restoration from texture preservation.
  • Network Architecture: A custom neural network is designed for this pipeline, featuring:
  • Dual Domain Prior Fusion (DDPF) module: Intelligently fuses the intensity and polarization priors, using mutual attention to handle outliers and bridge domain gaps.
  • Texture Guided Demodulation (TGD) modules: A series of cascaded modules that perform demodulation-like operations to extract high-quality texture guidance from the intermediate result (m) and apply it to the image reconstruction.

Datasets

The paper describes experiments on both synthetic and real-world datasets. A synthetic dataset was created for training and evaluation, and real-world images were captured for qualitative assessment. Specific details about the newly introduced synthetic dataset are available in the supplementary material.

Key Insights and Ideas

  • Polarization as a Shadow Cue: The paper demonstrates that polarization properties of light, specifically the degree of polarization (DoP), can serve as a powerful cue for shadow detection and removal, offering an alternative to traditional shadow masks.
  • Decoupling for Ill-Posedness: The two-stage pipeline is a key innovation. By first reconstructing a shadow-invariant representation (modulated intensity, m) and then using it to guide the final image reconstruction, the method effectively breaks down the highly ill-posed shadow removal problem into more manageable sub-problems.
  • Modulated Intensity (m) as an Intermediate Bridge: The concept of using m = d * I (intensity modulated by DoP) as an intermediate representation is crucial. It's shown to have a closer appearance between shadowed and non-shadowed regions compared to raw intensity, making it a more robust target for initial reconstruction.
  • Mutual Attention for Prior Fusion: The DDPF module's use of mutual attention to fuse intensity and polarization priors is a sophisticated way to leverage complementary information and mitigate the impact of noisy or outlier prior values.

Key Results

  • State-of-the-Art Performance: Pol-ShaRe consistently outperforms existing methods on both synthetic and real-world datasets across various metrics (PSNR, SSIM, RMSE). For instance, on synthetic data, it achieves a PSNR of 34.68 and SSIM of 0.990 in the shadow region, surpassing other methods.
  • Reduced Artifacts: Qualitative results show that Pol-ShaRe produces images with fewer artifacts, such as false colors and blurred textures, compared to mask-based and other mask-free methods.
  • Mask-Free Advantage: The method successfully removes shadows without requiring any shadow mask input, simplifying the workflow and improving robustness near shadow boundaries where masks are often inaccurate.

Limitations and Future Work

  • Prior Assumptions: The method's effectiveness relies on the assumptions made about the polarization properties of light. In scenarios where these assumptions don't hold (e.g., indoor artificial lighting, adverse weather), the polarization prior might introduce incorrect guidance.
  • Dynamic Scenes: Acquiring multiple polarized images for DoP estimation typically requires multiple shots. This limitation makes the method less suitable for highly dynamic scenes where image misalignment can occur.
  • Future Directions: The authors suggest that future work could explore handling scenarios where polarization priors are less reliable and investigating methods to mitigate issues arising from multi-shot acquisition in dynamic environments.

Summary updated

BibTeX
@inproceedings{zhou2025polarization,
  author = {Chu Zhou and Chao Xu and Boxin Shi},
  title = {{Polarization Guided Mask-Free Shadow Removal}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i10.33164},
}
Figure