Abstract
Shadow detection in sRGB images is often hindered by the Image Signal Processor (ISP), which compresses global contrast to preserve local details. This paper proposes a novel framework that detects shadows directly from RAW images using an Adaptive Illumination Mapping (AIM) module. By learning a many-to-many mapping that preserves multi-scale contrast and incorporating a shadow-aware feedback mechanism, the proposed method significantly outperforms existing sRGB-based detectors.
Introduction
Shadow detection is a critical component of many computer vision pipelines, yet it remains a challenging task. The primary difficulty lies in distinguishing shadows from dark, non-shadow objects, as both often exhibit similar local intensities. Traditional methods typically operate on sRGB images, which have already undergone significant processing within the camera's Image Signal Processor (ISP). This paper argues that the ISP itself is a major bottleneck and proposes a solution that leverages RAW image data directly.
The ISP Bottleneck
When a camera captures a scene, it records high-dynamic-range (HDR) RAW data. The internal ISP then converts this into a standard sRGB image. To create a visually pleasing result, ISPs often perform a "many-to-one" mapping that compresses global contrast to emphasize local details. While this enhances image aesthetics, it destroys the global intensity relationships that are essential for robust shadow detection. Consequently, shadows and dark surfaces become nearly indistinguishable in the sRGB domain.
Methodology: Adaptive Illumination Mapping
To overcome this limitation, the researchers developed a network architecture that bypasses the traditional ISP. The core of the approach is the Adaptive Illumination Mapping (AIM) module. Instead of a single output, the AIM module learns a "many-to-many" mapping, projecting linear RAW data into multiple sRGB images representing different intensity ranges. This ensures that contrast is preserved across multiple scales.
The system also includes a Shadow-Aware Feedback Mechanism. During training, initial shadow predictions are fed back into the AIM module. This forces the module to adaptively "re-expose" the generated images to maximize the visual difference between shadow and non-shadow regions, effectively creating a task-specific illumination enhancement.
The SARA Dataset
Because existing shadow detection datasets lack RAW data, the authors introduced the ShAdow RAw (SARA) dataset. It contains 7,019 pairs of RAW images and high-quality shadow masks, covering a wide variety of foreground objects and background scenes.
Key Results
The proposed method, also named SARA, achieves state-of-the-art performance. On its namesake dataset, it reached a Balanced Error Rate (BER) of 2.61, a significant improvement over the best sRGB-based methods (e.g., MTMT at 3.65) and even methods adapted for RAW data.
Qualitatively, the model excels in scenarios with high global contrast but low local contrast. Furthermore, the model is highly efficient, with an inference time of approximately 0.03 seconds on a standard GTX 3090 GPU, making it suitable for practical applications.
Conclusion
By shifting the focus from processed sRGB images to raw sensor data, this research highlights the importance of data integrity in low-level vision tasks. The adaptive illumination mapping approach provides a robust framework for handling high-dynamic-range scenes, with potential applications extending beyond shadow detection to areas like low-light enhancement and HDR imaging.
Summary updated