PP-Matting: High-Accuracy Natural Image Matting

Guowei Chen, Yi Liu, Jian Wang, Juncai Peng, Yuying Hao, Lutao Chu, Shiyu Tang, Zewu Wu, Zeyu Chen, Zhiliang Yu, Yuning Du, Qingqing Dang, Xiaoguang Hu, Dianhai YuarXiv 2022 · OpenReview

Abstract

Natural image matting is a fundamental and challenging computer vision task with numerous applications in image editing and composition. While deep learning has significantly advanced the field, most high-performing models require a user-supplied trimap, limiting real-world utility. Trimap-free approaches often suffer from foreground-background ambiguity and blurry details. This paper introduces PP-Matting, a trimap-free architecture that achieves high-accuracy results by employing a high-resolution detail branch (HRDB) and a semantic context branch (SCB). Extensive experiments on the Composition-1k and Distinctions-646 benchmarks demonstrate the superiority of PP-Matting over existing methods.

The Challenge of Natural Image Matting

Image matting involves precisely estimating the foreground opacity, or "alpha matte," for every pixel. Mathematically, an image $I$ is a combination of foreground $F$ and background $B$: $I = \alpha F + (1-\alpha)B$. Solving for $\alpha$ is highly ill-posed because we only observe $I$.

Traditionally, high-quality matting relies on a trimap—a manual guide labeling pixels as foreground, background, or unknown. However, creating trimaps is labor-intensive and impractical for real-time applications. Automated, trimap-free methods typically face two hurdles: semantic ambiguity (misidentifying the subject) and detail loss (blurring fine structures like hair).

Complex hair regions against busy backgrounds present significant challenges for automated matting algorithms.

The PP-Matting Architecture

PP-Matting is a single-stage, end-to-end architecture designed to eliminate the need for trimaps while maintaining high detail accuracy. Its design consists of three core components:

  • Shared Encoder (HRNet48): Unlike standard backbones that downsample images, HRNet maintains high-resolution representations throughout, preserving critical fine textures.
  • Semantic Context Branch (SCB): This branch performs a 3-class segmentation (foreground, background, and transition), providing the global understanding necessary to distinguish the subject from the background.
  • High-Resolution Detail Branch (HRDB): This branch focuses on extracting fine-grained details without changing feature resolution, preventing the loss of thin structures.

These branches are linked by a Guidance Flow mechanism using Gated Convolutional Layers (GCL). The SCB's semantic information guides the HRDB, ensuring the model focuses on transition areas—like the edges of hair—rather than the solid interior of the foreground.

Experimental Results

The model was evaluated on the Composition-1k and Distinctions-646 benchmarks, as well as a large-scale internal human matting dataset.

Quantitative Performance

On the Distinctions-646 dataset, PP-Matting achieved a Sum of Absolute Differences (SAD) of 40.69, a significant improvement over the previous state-of-the-art trimap-free method, HAttMatting (48.98). Notably, PP-Matting's performance on the Connectivity (Conn) metric even surpassed DIM, a well-known method that requires manual trimap guidance.

Qualitative Excellence

Visual evaluations confirm that PP-Matting excels at capturing difficult hair regions, producing clean, sharp edges that appear natural when composited onto new backgrounds.

Qualitative comparison showing PP-Matting's ability to preserve fine details compared to baseline methods.

Conclusion and Future Work

PP-Matting demonstrates that decoupling semantic context from detail extraction is a powerful strategy for trimap-free matting. By maintaining high resolution and using gated guidance, the model achieves professional-grade results. Future research may focus on reducing the computational overhead of the HRNet48 backbone for mobile applications and extending the architecture to handle highly transparent objects like glass or smoke.

Code and pre-trained models are available at PaddleSeg.

Summary updated

Abstract

Natural image matting is a fundamental and challenging computer vision task. It has many applications in image editing and composition. Recently, deep learning-based approaches have achieved great improvements in image matting. However, most of them require a user-supplied trimap as an auxiliary input, which limits the matting applications in the real world. Although some trimap-free approaches have been proposed, the matting quality is still unsatisfactory compared to trimap-based ones. Without the trimap guidance, the matting models suffer from foreground-background ambiguity easily, and also generate blurry details in the transition area. In this work, we propose PP-Matting, a trimap-free architecture that can achieve high-accuracy natural image matting. Our method applies a high-resolution detail branch (HRDB) that extracts fine-grained details of the foreground with keeping feature resolution unchanged. Also, we propose a semantic context branch (SCB) that adopts a semantic segmentation subtask. It prevents the detail prediction from local ambiguity caused by semantic context missing. In addition, we conduct extensive experiments on two well-known benchmarks: Composition-1k and Distinctions-646. The results demonstrate the superiority of PP-Matting over previous methods. Furthermore, we provide a qualitative evaluation of our method on human matting which shows its outstanding performance in the practical application. The code and pre-trained models will be available at PaddleSeg: https://github.com/PaddlePaddle/PaddleSeg.
BibTeX
@misc{chen2022pp,
  author = {Guowei Chen and Yi Liu and Jian Wang and Juncai Peng and Yuying Hao and Lutao Chu and Shiyu Tang and Zewu Wu and Zeyu Chen and Zhiliang Yu and Yuning Du and Qingqing Dang and Xiaoguang Hu and Dianhai Yu},
  title = {{PP-Matting: High-Accuracy Natural Image Matting}},
  year = {2022},
  eprint = {2204.09433},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2204.09433},
}
Figure