Abstract
Image matting is a fundamental task in computer vision, yet extracting text and WordArt remains a significant challenge due to intricate structures and small scale. We present ATM (Attentional Text Matting), a novel two-stage pipeline that utilizes a text detector as an attention mechanism. By cropping relevant regions before matting, ATM effectively handles small, disconnected, and complex character structures. Our approach significantly outperforms general-purpose matting algorithms, achieving a Sum of Absolute Differences (SAD) of 2.77 on our custom text matting dataset.
Introduction
Image matting involves separating a foreground object from its background by estimating pixel-level opacity (alpha). While significant progress has been made in portrait and general object matting, text matting has remained largely unexplored. Text, particularly stylized WordArt found in movie posters and advertisements, presents unique geometric and structural complexities that traditional models struggle to resolve.
The ability to accurately extract text allows for seamless re-composition in creative design, as shown in Figure 1, where a movie title is extracted and placed onto a new background without losing its stylistic essence.
The Challenges of Text Matting
Traditional matting algorithms often fail when applied to text due to three primary factors:
- Smallness: Text typically occupies a very small portion of a high-resolution image. Global networks often lose fine-grained details during downsampling.
- Multi-objectness: Unlike humans or cars, a single "word" is composed of multiple disconnected components (e.g., the dot on an 'i'). Models designed for continuous objects often fail to group these correctly.
- Complex Structures: WordArt frequently employs non-convex shapes, intricate shadows, and glowing effects that are far more complex than the relatively smooth contours of natural objects.
Methodology: The ATM Pipeline
To address these challenges, we propose a two-stage "attentional" pipeline. This architecture mimics the human visual system by first identifying a region of interest and then focusing on fine details.
Stage 1: Attentional Text Region Detector
The first stage employs a Connectionist Text Proposal Network (CTPN) to localize text regions. This acts as an attention mechanism, allowing the system to crop the original image into smaller patches. By "zooming in" on these patches, the subsequent matting network receives a much higher signal-to-noise ratio per pixel.
Stage 2: Trimap and Matte Generation
The cropped patches are processed by a dual-network structure:
- T-Net: This network generates a "trimap," which segments the image into foreground, background, and an "unknown" transition region.
- M-Net: This network focuses on capturing fine details to generate the final alpha matte.
- Fusion Module: A specialized layer blends the outputs of T-Net and M-Net, ensuring the final result maintains global structural integrity while preserving sharp edge details.
Post-processing
Once individual patches are processed, they must be integrated back into the full-resolution image. We introduce Matte Ranking and Matte Averaging techniques to stitch these patches together seamlessly, ensuring consistency across the entire high-resolution output.
Data Synthesis
A major hurdle in text matting research was the lack of a dedicated dataset. To solve this, we developed a Text Matting Image Synthesis Engine. This engine generates realistic training data by sampling diverse fonts, colors, and textures, then blending them with clean backgrounds using Poisson Image Editing. Our dataset contains over 46,000 unique foregrounds, providing the necessary variety to train a robust model.
Results and Discussion
We evaluated ATM against nine state-of-the-art matting methods. Quantitatively, ATM achieved a Sum of Absolute Differences (SAD) of 2.77, which is nearly a 50% improvement over the next best automatic method, Semantic Human Matting (SHM), which scored 5.62.
Qualitatively, ATM excels at capturing the "fuzziness" of glowing text and the sharp internal holes of complex characters. This precision is critical for professional design applications where even minor artifacts can ruin a composition.
Conclusion
ATM demonstrates that treating detection as an attention mechanism is a powerful strategy for matting small, complex objects. By modularizing the pipeline, the system remains flexible; the detector can be swapped to handle different text orientations or languages without retraining the core matting networks. Future work will focus on expanding this robustness to multi-oriented and multilingual text.
Summary updated