ATM: Attentional Text Matting

Peng Kang, Jianping Zhang, Chen Ma, Guiling Sun

Abstract

Image matting is a fundamental task in computer vision, yet extracting text and WordArt remains a significant challenge due to intricate structures and small scale. We present ATM (Attentional Text Matting), a novel two-stage pipeline that utilizes a text detector as an attention mechanism. By cropping relevant regions before matting, ATM effectively handles small, disconnected, and complex character structures. Our approach significantly outperforms general-purpose matting algorithms, achieving a Sum of Absolute Differences (SAD) of 2.77 on our custom text matting dataset.

Introduction

Image matting involves separating a foreground object from its background by estimating pixel-level opacity (alpha). While significant progress has been made in portrait and general object matting, text matting has remained largely unexplored. Text, particularly stylized WordArt found in movie posters and advertisements, presents unique geometric and structural complexities that traditional models struggle to resolve.

Application of text matting in a movie poster re-composition workflow.

The ability to accurately extract text allows for seamless re-composition in creative design, as shown in Figure 1, where a movie title is extracted and placed onto a new background without losing its stylistic essence.

The Challenges of Text Matting

Traditional matting algorithms often fail when applied to text due to three primary factors:

  1. Smallness: Text typically occupies a very small portion of a high-resolution image. Global networks often lose fine-grained details during downsampling.
  2. Multi-objectness: Unlike humans or cars, a single "word" is composed of multiple disconnected components (e.g., the dot on an 'i'). Models designed for continuous objects often fail to group these correctly.
  3. Complex Structures: WordArt frequently employs non-convex shapes, intricate shadows, and glowing effects that are far more complex than the relatively smooth contours of natural objects.

Methodology: The ATM Pipeline

To address these challenges, we propose a two-stage "attentional" pipeline. This architecture mimics the human visual system by first identifying a region of interest and then focusing on fine details.

The ATM pipeline architecture, featuring an attentional detector followed by a dual-network matting system.

Stage 1: Attentional Text Region Detector

The first stage employs a Connectionist Text Proposal Network (CTPN) to localize text regions. This acts as an attention mechanism, allowing the system to crop the original image into smaller patches. By "zooming in" on these patches, the subsequent matting network receives a much higher signal-to-noise ratio per pixel.

Stage 2: Trimap and Matte Generation

The cropped patches are processed by a dual-network structure:

  • T-Net: This network generates a "trimap," which segments the image into foreground, background, and an "unknown" transition region.
  • M-Net: This network focuses on capturing fine details to generate the final alpha matte.
  • Fusion Module: A specialized layer blends the outputs of T-Net and M-Net, ensuring the final result maintains global structural integrity while preserving sharp edge details.

Post-processing

Once individual patches are processed, they must be integrated back into the full-resolution image. We introduce Matte Ranking and Matte Averaging techniques to stitch these patches together seamlessly, ensuring consistency across the entire high-resolution output.

Data Synthesis

A major hurdle in text matting research was the lack of a dedicated dataset. To solve this, we developed a Text Matting Image Synthesis Engine. This engine generates realistic training data by sampling diverse fonts, colors, and textures, then blending them with clean backgrounds using Poisson Image Editing. Our dataset contains over 46,000 unique foregrounds, providing the necessary variety to train a robust model.

Results and Discussion

We evaluated ATM against nine state-of-the-art matting methods. Quantitatively, ATM achieved a Sum of Absolute Differences (SAD) of 2.77, which is nearly a 50% improvement over the next best automatic method, Semantic Human Matting (SHM), which scored 5.62.

Qualitatively, ATM excels at capturing the "fuzziness" of glowing text and the sharp internal holes of complex characters. This precision is critical for professional design applications where even minor artifacts can ruin a composition.

Conclusion

ATM demonstrates that treating detection as an attention mechanism is a powerful strategy for matting small, complex objects. By modularizing the pipeline, the system remains flexible; the detector can be swapped to handle different text orientations or languages without retraining the core matting networks. Future work will focus on expanding this robustness to multi-oriented and multilingual text.

Summary updated

Abstract

Image matting is a fundamental computer vision problem and has many applications. Previous image matting methods always focus on extracting a general object or portrait from the background in an image. In this paper, we try to solve the text matting problem, which extracts characters (usually WordArts) from the background in an image. Different from traditional image matting problems, text matting is much harder because of its foreground's three properties: smallness, multi-objectness, and complicated structures and boundaries. We propose a two-stage attentional text matting pipeline to solve the text matting problem. In the first stage, we utilize text detection methods to serve as the attention mechanism. In the second stage, we employ the attentional text regions and matting system to obtain mattes of these text regions. Finally, we post-process the mattes and obtain the final matte of the input image. We also construct a large-scale dataset with high-quality annotations consisting of 46,289 unique foregrounds to facilitate the learning and evaluation of text matting. Extensive experiments on this dataset and real images clearly demonstrate the superiority of our proposed pipeline over previous image matting methods on the task of text matting.
BibTeX
@inproceedings{kang2021atm,
  author = {Peng Kang and Jianping Zhang and Chen Ma and Guiling Sun},
  title = {{ATM: Attentional Text Matting}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2021},
  doi = {10.1109/wacv48630.2021.00395},
}

Loading related content...

Figure