Abstract
Text removal is a critical task in image processing, essential for privacy protection, document restoration, and augmented reality. However, accurately reconstructing complex backgrounds after removing text remains a significant challenge. This paper introduces the Contextual-guided Text Removal Network (CTRNet), a novel framework that decouples the text removal process into specialized components. By integrating low-level structural priors, high-level semantic context, and a hybrid CNN-Transformer architecture for local-global content modeling, CTRNet achieves state-of-the-art performance in restoring high-fidelity backgrounds.
Introduction
Text removal is more than a simple erasure task; it is a complex image-to-image translation problem that requires sophisticated inpainting capabilities. While existing methods, particularly those based on Generative Adversarial Networks (GANs), have made strides in identifying and removing text, they frequently struggle when the background behind the text contains intricate textures or complex structural patterns.
A primary limitation of current models is their attempt to map raw images directly to text-erased counterparts in a single step. This approach often results in an imbalance between the "erasing" and "restoring" phases, leading to blurry artifacts or inconsistent textures. To address this, a model must possess a robust "memory" of the background's local textures and its global structural consistency. CTRNet addresses these issues by modeling local-global context to ensure seamless background recovery.
Methodology
CTRNet decouples the text removal process into four specialized components designed to handle different aspects of the restoration task:
1. Text Perception Head
Unlike traditional methods that utilize "hard" binary masks (0 or 1), CTRNet employs a Text Perception Head to generate a Soft Mask. This approach creates a smoother transition between text and non-text regions, significantly reducing edge artifacts and allowing for better blending during the restoration phase.
2. Low-level Contextual Guidance (LCG)
The LCG block focuses on the "skeleton" or low-frequency structure of the background. Utilizing Relative Total Variation (RTV), it extracts structural priors that guide the model on the basic shapes and layout of the scene, independent of fine-grained textures. This structural map serves as a reliable foundation for the subsequent reconstruction.
3. High-level Contextual Guidance (HCG)
To provide a global understanding of the scene, the HCG block utilizes a pretrained TResNet classifier to extract semantic features. This high-level context ensures that the restored regions are semantically consistent with the overall image—for instance, ensuring that a restored patch on a building facade matches the architectural style of the rest of the structure.
4. Local-Global Content Modeling (LGCM)
The LGCM is the core of the CTRNet architecture. It utilizes a hybrid approach:
- CNNs are employed to capture local textures and fine details.
- Transformer-Encoders are used to establish long-range dependencies between distant pixels.
This dual modeling ensures that the network can leverage patterns from distant parts of the image to fill in missing regions, ensuring both local detail and global coherence.
Experimental Results
Datasets
The model was rigorously evaluated on three distinct benchmarks:
- SCUT-Syn: A synthetic dataset of 8,800 images where text is digitally overlaid on natural backgrounds.
- SCUT-EnsText: A challenging real-world dataset comprising 3,562 images with manually erased text, sourced from public benchmarks like ICDAR and MS COCO-Text.
- Handwritten Document Dataset: 1,000 images of examination papers were used to test the model's generalizability to document restoration tasks.
Quantitative Performance
CTRNet demonstrated superior performance over previous state-of-the-art (SOTA) methods, such as EraseNet and PERT, across multiple metrics:
- SCUT-EnsText: The model achieved a PSNR of 35.20 and an FID of 13.99, outperforming the previous best PSNR by nearly 2 points.
- SCUT-Syn: Performance reached a PSNR of 41.28, a significant jump from the previous SOTA of 39.40.
Qualitative Analysis
Visual comparisons highlight CTRNet's ability to restore sharp, believable textures in complex scenarios where competing models often produce "smudged" or "melted" results. Whether dealing with patterned lanterns or brick walls, CTRNet consistently maintains structural integrity and textural detail.
Discussion and Conclusion
CTRNet represents a significant advancement in text removal by emphasizing the importance of local-global context and structural priors. The shift from hard masks to soft masks and the integration of Transformers for long-range dependency modeling are key contributors to its success.
Despite its performance, the model's reliance on an 8-stage Transformer-based LGCM block suggests a higher computational overhead compared to simpler CNN architectures. Future work may focus on optimizing inference speed and memory usage. Additionally, while the model excels at scene text removal, extremely dense document text remains a frontier for further refinement. Ultimately, the local-global modeling approach pioneered in CTRNet holds promise for broader image restoration tasks, including general object removal and watermark cleaning.
Summary updated