Abstract
Image shadow removal is a spatially heterogeneous problem where degradation levels vary significantly across an image. Conventional deep learning models relying on weight sharing often struggle to specialize for both shadowed and non-shadowed regions simultaneously. We introduce HomoFormer, a novel architecture that employs a "homogenization" strategy. By applying a stochastic random shuffle operation, HomoFormer distributes non-uniform shadow degradation across the entire image, allowing standard local window Transformers to operate on a homogenized data distribution. To preserve spatial context, we integrate a Structure Modeling MLP (SMLP) that restores local relationships. Our approach achieves state-of-the-art results on major benchmarks, including ISTD+ and SRD.
Introduction: The Challenge of Spatial Heterogeneity
Shadow removal requires handling diverse patterns: some regions are totally shadowed, while others are partially shadowed or clear. Most dominant architectures, including CNNs and window-based Transformers (e.g., Swin Transformer), utilize weight sharing. This forces the model to find an unsatisfactory compromise between parameters suited for dark shadow cores and those for soft edges. While global Transformers could theoretically address this, their quadratic computational complexity makes them impractical for high-resolution restoration.
Methodology: The Homogenized Transformer
HomoFormer addresses non-uniformity through a three-step homogenization process within its core blocks:
- Random Shuffle ($S$): Pixel features are stochastically rearranged across the spatial domain. This spreads concentrated shadow patches thinly and evenly across the entire feature map.
- Local Self-Attention: The homogenized features are processed by a standard local window-based self-attention layer. Because self-attention is permutation equivariant, it can effectively process shuffled tokens without losing information. This mechanism allows local windows to capture sparse global interactions, as each window now contains a random sample of pixels from the whole image.
- Inverse Shuffle ($I_S$): Features are returned to their original spatial positions to restore the image's semantic structure.
To compensate for the loss of local structural information during shuffling, the authors replace the standard Feed-Forward Network (FFN) with a Structure Modeling MLP (SMLP). This module utilizes depth-wise convolutions to re-learn spatial relationships that the shuffling step temporarily discards.
Experimental Results
We evaluated HomoFormer on two primary benchmarks:
- Adjusted ISTD (ISTD+): HomoFormer achieved a Mean Absolute Error (MAE) of 2.6 for the whole image, outperforming previous leaders like ShadowDiffusion (2.7) and BMNet (3.0).
- SRD Dataset: The model reached a PSNR of 35.37 dB, a significant improvement over the previous state-of-the-art, ShadowFormer (32.90 dB).
Qualitative analysis indicates that HomoFormer produces cleaner results with fewer boundary artifacts compared to existing methods. Furthermore, the stochastic nature of the shuffle allows for uncertainty estimation; the variance across multiple runs correlates with potential model errors, providing an implicit confidence map.
Conclusion and Future Work
HomoFormer demonstrates that "homogenizing" data to fit simpler models can be more effective than building increasingly complex adaptive architectures. While the current implementation uses Monte Carlo averaging (HomoFormer+) to mitigate inference stochasticity, the underlying philosophy of homogenization holds promise for other spatially heterogeneous tasks, such as image inpainting and non-uniform deblurring.
Summary updated