TL;DR
STARS (Stage-Adaptive Responsibility Separation) is a new framework that solves a persistent problem in shadow removal: the accidental "brightness shift" in non-shadowed regions. By decoupling the restoration process into a three-stage cascade—where the final stage is explicitly tasked with preserving non-shadow areas—and leveraging foundation model priors via a novel adapter, the authors achieve state-of-the-art results across both remote sensing and natural image datasets.
What Problem Does This Paper Address?
In remote sensing, shadows aren't just an aesthetic issue; they hide critical information for object detection and land-use classification. Most modern shadow removal models use a "mask prior"—a binary map telling the network where the shadows are.
However, the authors identify a frustrating trade-off: when a single network is trained to restore shadows, it often inadvertently "modulates" the non-shadow regions, causing a systematic shift in brightness. In remote sensing, where non-shadow regions typically occupy the vast majority of the image, even a tiny shift in these areas leads to a massive drop in overall image quality metrics (like PSNR). The fundamental challenge is that "restoring a shadow" and "leaving a non-shadow region unchanged" are conflicting tasks that compete for the same neural weights during optimization.
Methods
The authors propose a "task-decoupling" perspective realized through three main components:
- Stage-Adaptive Responsibility Separation (STARS): Instead of one big network, they use a three-stage cascade of "ShadowFormer" backbones.
- Stages 1 & 2 are fed the shadow mask ($M$) and focus on restoring the shadowed pixels.
- Stage 3 is fed the inverse mask ($1-M$). Its explicit responsibility is to correct any brightness shifts or artifacts introduced to the non-shadow regions by the previous stages.
- Feature Selection and Fusion Adapter (FSFA): To bring high-level "common sense" into the model, the authors tap into DINOv3, a large-scale vision foundation model. Because DINOv3 was trained on natural images and remote sensing data is different, the FSFA calibrates these features using per-image statistics and learnable weights to select the most useful layers (balancing mid-level details with deep semantics).
- Region-Specific Loss ($\mathcal{L}_{ns}$): In the final stage, the model uses a specific L1 loss that only calculates gradients for non-shadow pixels. This ensures the optimization signal for the final stage is purely focused on its assigned responsibility: identity mapping for non-shadow regions.
Datasets
The model was evaluated on three challenging benchmarks:
- UAV-SC+: A specialized remote sensing dataset captured by unmanned aerial vehicles.
- ISTD+: A classic, color-corrected benchmark for shadow removal in natural, everyday scenes.
- WSRD+ 2026: The benchmark used for the CVPR 2026 NTIRE Image Shadow Removal Challenge, representing the current "gold standard" for the field.
Key Insights and Ideas
- Structural Decoupling over Joint Optimization: The most important insight is that regional heterogeneity (shadow vs. non-shadow) requires structural separation. You can't just ask one set of weights to do two opposite things at once; you have to assign those tasks to different stages of a pipeline.
- The Mask as a Gate, Not Just an Input: In STARS, the mask isn't just a hint; it acts as a structural gate ($M$ vs $1-M$) that defines the "responsibility" of each stage in the cascade.
- Foundation Models for Restoration: The paper demonstrates that even though foundation models like DINOv3 aren't trained for restoration, their internal representations of "what an object should look like" provide invaluable guidance for filling in shadowed areas, provided they are properly adapted (via FSFA).
Key Results
STARS outperformed all existing methods across all three datasets:
- UAV-SC+: Achieved a whole-image PSNR of 26.65 dB, significantly outperforming the previous best (RASM at 24.63 dB). Crucially, it improved non-shadow region PSNR to 28.08 dB, proving it successfully mitigated the brightness shift.
- ISTD+: Set a new state-of-the-art with a shadow-region PSNR of 38.72 dB and a non-shadow PSNR of 37.47 dB, showing the method works just as well for natural photos as it does for satellite imagery.
- WSRD+ 2026: STARS beat the winning solution of the CVPR 2026 NTIRE challenge by 0.83 dB in overall PSNR and significantly improved perceptual metrics (SSIM and LPIPS).
Limitations and Future Work
The primary drawback of the STARS framework is its computational cost. Because it relies on a three-stage cascade and a large frozen foundation model (DINOv3) for guidance, it requires more memory and time than single-stage models.
The authors suggest that future work should focus on efficiency, perhaps by exploring parameter-sharing between the stages or developing lighter versions of the semantic guidance adapter to make the model more practical for real-time remote sensing applications.
Summary updated