Mitigating Intensity Bias in Shadow Detection via Feature Decomposition and Reweighting

Lei Zhu, Ke Xu, Zhanghan Ke, Rynson W. H. Lau

Abstract

Shadow detection is a fundamental task in computer vision, yet deep learning models often suffer from "intensity bias," over-relying on image darkness as a primary cue. This leads to significant errors in dark non-shadow regions and bright shadow areas. We present a novel framework centered on a Feature Decomposition and Reweighting (FDR) module that explicitly separates intensity-variant features from intensity-invariant semantic features. By utilizing a dual self-supervised training strategy and a cumulative learning schedule, our proposed FDRNet forces the network to prioritize robust semantic information over simple intensity cues. Experimental results on SBU, UCF, and ISTD benchmarks demonstrate that FDRNet achieves state-of-the-art performance and exhibits superior robustness to brightness variations.

Introduction

Despite the success of Convolutional Neural Networks (CNNs) in shadow detection, they frequently exhibit a failure mode termed "intensity bias." Because shadows are inherently darker than their surroundings, models often learn a simplistic "dark = shadow" heuristic. This results in two primary issues: False Positives, where dark objects (e.g., black cars) are misidentified as shadows, and False Negatives, where bright regions within a shadow (e.g., white lane markings) are overlooked.

Furthermore, these models often lack consistency; minor global brightness shifts can drastically alter predicted shadow masks. This paper argues that networks are "lazy" and will prioritize local intensity features if they provide a sufficient training signal. To address this, we propose a method to decouple these features and systematically de-prioritize intensity information during training.

Methodology

Feature Decomposition and Reweighting (FDR)

The core of our architecture is the FDR module, which splits the feature map $F$ into two distinct streams:

  1. Intensity-Variant ($F_v$): Captures brightness-related information.
  2. Intensity-Invariant ($F_i$): Captures semantics, edges, and connectivity, which should remain stable under varying illumination.
Overview of the FDRNet architecture including the Feature Decomposition and Reweighting module and auxiliary self-supervised tasks.

Dual Self-Supervised Tasks

To ensure effective feature separation, we employ a bilateral training branch. Given an image $I$ and a brightness-shifted version $I'$, we apply two auxiliary losses:

  • Invariant Loss ($\mathcal{L}_i$): Ensures that $F_i$ extracted from both $I$ and $I'$ are identical.
  • Variant Loss ($\mathcal{L}_v$): Forces $F_v$ to predict the magnitude of the applied brightness shift, ensuring the variant stream accurately tracks intensity changes.

Cumulative Learning Strategy

We combine features using a trade-off parameter $\mu$. Early in training, $\mu$ is high, allowing the model to utilize easy intensity cues. As training progresses, $\mu$ follows a parabolic decay, forcing the network to "drop" intensity information and rely on the more robust invariant features. This acts as a targeted "soft-dropout" for intensity-based information.

Experimental Results

Quantitative Performance

FDRNet was evaluated on three standard benchmarks: SBU, UCF, and ISTD. Using the Balanced Error Rate (BER) metric (lower is better), our model achieved state-of-the-art results:

  • SBU: 3.04 BER (vs. previous best 3.15).
  • UCF: 7.28 BER (vs. previous best 7.47).
  • ISTD: 1.55 BER (vs. previous best 1.72).

Qualitative Analysis

Qualitative comparisons demonstrate that FDRNet is significantly more robust than previous methods. It successfully avoids false positives on dark objects and maintains mask integrity across bright markings within shadows.

Qualitative comparison of FDRNet against state-of-the-art shadow detection methods across various challenging scenes.

Conclusion

By explicitly decoupling intensity-variant and invariant features, FDRNet mitigates the inherent intensity bias in shadow detection models. While the model shows exceptional performance in daytime scenarios, future work will focus on domain adaptation to improve performance in night-time scenes, where the statistics of illumination differ significantly from standard training datasets.

Summary updated

Abstract

Although CNNs have achieved remarkable progress on the shadow detection task, they tend to make mistakes in dark non-shadow regions and relatively bright shadow regions. They are also susceptible to brightness change. These two phenomenons reveal that deep shadow detectors heavily depend on the intensity cue, which we refer to as intensity bias. In this paper, we propose a novel feature decomposition and reweighting scheme to mitigate this intensity bias, in which multi-level integrated features are decomposed into intensity-variant and intensity-invariant components through self-supervision. By reweighting these two types of features, our method can reallocate the attention to the corresponding latent semantics and achieves balanced exploitation of them. Extensive experiments on three popular datasets show that the proposed method outperforms state-of-the-art shadow detectors.
BibTeX
@inproceedings{zhu2021mitigating,
  author = {Lei Zhu and Ke Xu and Zhanghan Ke and Rynson W. H. Lau},
  title = {{Mitigating Intensity Bias in Shadow Detection via Feature Decomposition and Reweighting}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2021},
  doi = {10.1109/iccv48922.2021.00466},
}
Figure