Abstract: Existing works on document image shadow removal mostly depend on learning and leveraging a constant background (the color of the paper) from the image. However, the constant background is less representative and frequently ignores other background colors, such as the printed colors, resulting in distorted results. In this paper, we present a color-aware background extraction network (CBENet) for extracting a spatially varying background image that accurately depicts the background colors of the document. Furthermore, we propose a background-guided document images shadow removal network (BGShadowNet) using the predicted spatially varying background as auxiliary information, which consists of two stages. At Stage I, a background-constrained decoder is designed to promote a coarse result. Then, the coarse result is refined with a background-based attention module (BAModule) to maintain a consistent appearance and a detail improvement module (DEModule) to enhance the texture details at Stage II. Experiments on two benchmark datasets qualitatively and quantitatively validate the superiority of the proposed approach over state-of-the-arts.
The Challenge of Complex Document Backgrounds
Shadow removal is a critical preprocessing step for digitizing documents, especially those captured via mobile phones where lighting is often uneven. While natural image shadow removal has progressed significantly, these methods often struggle with documents because they prioritize text content over background textures.
Existing document-specific methods typically assume the background (the paper color) is a single, constant value. However, modern documents—such as leaflets, textbooks, and receipts—often feature colored graphics, varied paper tones, or complex layouts. When a model assumes a constant background, it often results in "shadow remnants" or severe color distortion in areas where the actual background color deviates from the average. This research addresses this by treating the background as a spatially varying map that guides the restoration process.
Methodology: CBENet and BGShadowNet
The proposed framework consists of two main components designed to handle background complexity:
1. CBENet (Color-aware Background Extraction Network)
This network predicts a "shadow-free, text-free" version of the document. Instead of a single color value, it generates a full-resolution background map using a U-Net structure. To train it, the authors developed a "background calculator" that uses Gaussian Mixture Models (GMM) to separate text from background in local patches, followed by guided filtering to smooth boundaries while preserving color edges.
2. BGShadowNet (Background-guided Shadow Removal Network)
This two-stage architecture uses the predicted background as a roadmap for restoration:
- Stage I: A background-constrained decoder fuses image features with the predicted background features to produce a coarse, shadow-free result.
- Stage II (Refinement): This stage uses two specialized modules: the BAModule (Background-based Attention Module), which ensures color and illumination consistency, and the DEModule (Detail Enhancement Module), which uses "feature equalization" to recover fine texture details and sharpen text edges.
Experimental Results
The study utilizes two primary datasets: the Real Document Dataset (RDD), containing 4,916 pairs of real-world images, and Kligler’s Dataset for cross-evaluation.
Quantitatively, on the RDD dataset, BGShadowNet achieved an RMSE of 2.219 and a PSNR of 37.585, significantly outperforming the next best competitor, BEDSR-Net (2.937 RMSE and 34.928 PSNR). Qualitatively, visual comparisons show that while other models leave "ghost" shadows or turn colored sections gray, BGShadowNet maintains vibrant colors and clean, legible text. In a blind user study with 100 volunteers, the proposed results were chosen as the best shadow-free images more frequently than seven other established methods.
Conclusion
By moving away from the "constant background" assumption and utilizing a spatially varying background as a prior, BGShadowNet effectively removes shadows from complex documents without the color distortion common in previous methods. While heavy sensor noise remains a challenge for future work, this approach sets a new state-of-the-art for document restoration in real-world conditions.
Summary updated