Abstract
Shadow removal remains a significant challenge in computer vision, primarily due to the difficulty of obtaining paired datasets for supervised learning. This paper presents Boundary-aware Conditional Diffusion (BCDiff), a novel unsupervised framework that leverages the generative power of pre-trained diffusion models. By adopting a "divide and conquer" strategy that separately treats shadow, non-shadow, and complex boundary regions, the proposed method achieves state-of-the-art unsupervised performance, rivaling many fully supervised alternatives.
Introduction
Deep learning has significantly advanced shadow removal, yet most high-performing models rely on supervised learning with paired "shadow/shadow-free" image datasets. Such datasets are laborious to collect and often fail to generalize to diverse real-world conditions. Unsupervised methods, while more flexible, frequently suffer from "hallucinations"—the creation of artifacts or incorrect textures—due to a lack of strong natural image priors.
A key challenge in shadow removal is the non-uniform nature of shadows, which consist of a dark "umbra" (the center) and a blurry "penumbra" (the boundary). Standard models often struggle to address both simultaneously. BCDiff addresses this by treating shadow removal as a localized restoration problem guided by global image priors and a specific focus on boundary transitions.
Methodology: Boundary-aware Conditional Diffusion (BCDiff)
The BCDiff framework is built upon three core pillars designed to provide structural and contextual guidance during the restoration process.
1. Pre-trained Diffusion Prior
Instead of training a model from scratch, the method utilizes a pre-trained unconditional diffusion model. This model acts as a powerful "natural image prior," ensuring that the generated shadow-free regions remain visually coherent and realistic.
2. Context Conditioned Generation (CCG)
During the iterative denoising process, the model uses non-shadow regions as a context. It effectively "inpaints" the shadow areas by referencing the surrounding textures and lighting conditions. This ensures that the restored area doesn't just look like a smooth patch but has the correct underlying detail.
3. Shadow-Invariant Intrinsic Decomposition (SIID)
This "Divide and Conquer" module decomposes the image into reflectance (material property) and illumination. While illumination varies across a shadow boundary, reflectance should remain constant. By enforcing reflectance consistency, the model can recover fine structural details in dark regions that might otherwise be lost. This solves the "isolation" problem, where the center of a large shadow lacks the immediate context needed for accurate restoration.
Experiments and Results
The researchers evaluated BCDiff on the ISTD Dataset and a Video Shadow Removal Dataset. Despite being unsupervised and training-free for the removal task itself, the method demonstrated remarkable performance.
Quantitative Performance
On the ISTD benchmark, BCDiff achieved a PSNR of 35.91 in shadow regions, a significant improvement over the previous unsupervised state-of-the-art, G2R (33.58 PSNR). Remarkably, it also outperformed several supervised models in terms of Root Mean Square Error (RMSE) and Structural Similarity Index (SSIM).
Qualitative Analysis
Visually, the method excels at producing clean boundaries. While competing unsupervised models often leave "ghost" edges or introduce color tints, BCDiff's results are nearly indistinguishable from ground truth. The model effectively handles the mathematical complexity of the penumbra while applying linear shifts to the simpler umbra regions.
Conclusion
BCDiff represents a significant step forward for unsupervised shadow removal. By combining the generative power of diffusion models with a boundary-aware "divide and conquer" strategy, it overcomes the common pitfalls of GAN-based unsupervised methods. While the method currently faces limitations regarding computational speed (requiring 250 timesteps) and a dependency on shadow masks, its ability to match supervised performance without paired training data marks a major milestone in the field.
Summary updated