Image matting is the process of precisely separating a foreground object from its background. Unlike simple binary segmentation, matting accounts for "soft" edges—the semi-transparent pixels found in hair, fur, or glass. Mathematically, this is an under-constrained problem: for every pixel, we must solve the equation $I = \alpha F + (1-\alpha)B$, where we only know the observed color $I$ and must guess the foreground $F$, background $B$, and the transparency $\alpha$.
To solve this, researchers traditionally relied on a trimap: a manually drawn guide that labels pixels as definitely foreground, definitely background, or "unknown." However, as noted in Deep Image Matting by Ning Xu, this manual step is the primary bottleneck for the field. Recent research focuses on eliminating this "trimap tax" through sparse interaction, semantic automation, and the use of large-scale foundation models.
From Brushes to Clicks: Sparse Interaction
The most direct way to reduce labor is to replace dense hand-painted trimaps with sparse user inputs. Instead of tracing every strand of hair, can a model infer the trimap from a few clicks?
Xin Yang and colleagues introduced Active Matting, which uses reinforcement learning to "ask" the user for labels only in the most informative regions. Similarly, dugMatting by Jiawei Wu decomposes uncertainty into model ignorance and data noise. It uses these signals to prompt users for targeted clicks, achieving professional results with a fraction of the effort. Yihan Hu further streamlined this in Click2Trimap, treating trimap generation as a specialized three-class segmentation task that can be completed in seconds.
Semantic Guidance: The Model as Annotator
If a model understands what it is looking at, it shouldn't need a human to tell it where the edges are. This "trimap-free" or "automatic" matting relies on high-level semantic cues to constrain the problem.
Yu Qiao developed HAttMatting, which uses a dual-attention mechanism to aggregate global semantic "profiles" with low-level texture details. This allows the model to "glance" at the object to understand its shape and "focus" on the fine details like fur. Yuhongze Zhou proposed a similar two-stage pipeline in Human Perception Modeling for Automatic Natural Image Matting, where a dedicated network generates an internal trimap before passing it to a refinement module. For specific domains like portraits, Jiake Xie showed in SDNet that self-distillation can create extremely lightweight models capable of automatic matting on mobile devices.
Temporal Propagation: Matting Through Time
Video matting compounds the trimap problem: annotating 30 frames per second is impossible for most workflows. The solution is to propagate information from a single annotated frame across the entire sequence.
In One-Trimap Video Matting, Hongje Seong and Seoung Wug Oh demonstrated that a single user-annotated frame can guide an entire video by jointly modeling trimap propagation and alpha prediction. Bo Xu extended this logic in HSTSG, using hierarchical memory matching to maintain temporal stability. Even without any manual input, MODNet-V by Jiayu Sun achieves video matting by "recovering" the background directly from the video stream, using it as a reference to resolve foreground ambiguity.
Leveraging Foundation Models
The most recent shift involves "grafting" the zero-shot capabilities of large vision models onto the matting task. These models provide a powerful semantic prior that can handle "anything," not just humans or animals.
Jiachen Li introduced the Matting Anything Model (MAM), which pairs the Segment Anything Model (SAM) with a lightweight refinement module. This allows users to extract complex objects using simple text prompts or boxes. Longfei Huang took this further with SDMatte, which uses the generative priors of Stable Diffusion to "hallucinate" fine details like hair and transparency that are often lost in traditional discriminative models. By moving from manual painting to high-level "in-context" instructions—as seen in IconMatting by He Guo—the field is successfully decoupling high-quality results from high-intensity labor.