Abstract
Image matting is a fundamental technique in image and video editing, yet traditional deep learning approaches often struggle with high-resolution (HR) inputs due to hardware memory constraints. This paper introduces HDMatt, the first deep learning-based framework specifically designed for high-resolution image matting. By employing a novel "crop-and-stitch" strategy integrated with a Cross-Patch Contextual (CPC) module, HDMatt effectively captures both fine-grained local details and global contextual dependencies. Experimental results on the Adobe Image Matting and AlphaMatting benchmarks demonstrate that HDMatt achieves state-of-the-art performance, particularly on real-world ultra-high-resolution images.
The Problem: The "Memory Wall" in Matting
Precise foreground extraction, or alpha matting, is critical for high-quality composition. While deep learning has significantly improved matting accuracy, modern high-resolution images (e.g., 6000x6000px) exceed the memory capacity of standard GPUs. Researchers typically resort to two suboptimal workarounds:
- Downsampling: Reducing image size to fit memory, which destroys the fine details (hair, fur, translucent edges) that matting is intended to preserve.
- Naive Cropping: Processing the image in independent patches. This leads to a loss of global context; without seeing the broader scene, a model may fail to distinguish between similar foreground and background textures, resulting in inconsistent artifacts across patch boundaries.
Methodology: HDMatt and the CPC Module
HDMatt addresses these issues by processing images in a patch-based manner while maintaining global awareness through a specialized architecture.
Encoder-Decoder with ASPP
The core architecture utilizes a ResNet-34 backbone. It incorporates Atrous Spatial Pyramid Pooling (ASPP) to extract multi-scale features, ensuring the model can handle objects of varying sizes within patches.
Cross-Patch Contextual (CPC) Module
The CPC module is the primary innovation of HDMatt. Instead of treating each patch in isolation, the CPC module allows a "query patch" to sample and attend to "reference patches" from across the entire image. By calculating similarity scores, the model identifies the most relevant context to inform the matting process in the current patch.
Trimap-Guided Non-Local (TGNL) Operation
To further refine the context, HDMatt uses the provided trimap (which labels regions as foreground, background, or unknown). The TGNL operation explicitly propagates information from known foreground and background regions in reference patches to the unknown regions of the query patch, ensuring structural and color consistency.
Evaluation and Results
HDMatt was evaluated using the Adobe Image Matting (AIM) dataset, AlphaMatting.com, and a new HR Real-World Dataset featuring images up to 6000x6000px.
Key Findings
- State-of-the-Art Accuracy: HDMatt achieved the top rank on the AlphaMatting.com benchmark at the time of submission, leading in SAD (Sum of Absolute Differences), MSE, and Gradient metrics.
- Quantitative Performance: On the AIM test set, HDMatt achieved a SAD of 33.5, outperforming previous leaders like ContextNet (35.8).
- Visual Fidelity: As shown in the comparative results, HDMatt preserves intricate details that are often blurred by downsampling-based methods or rendered inconsistently by naive patch-based approaches.
Conclusion and Future Work
HDMatt successfully bridges the gap between deep learning matting models and high-resolution real-world requirements. While the model sets a new standard, the authors note that performance is still limited by the resolution and detail of existing training datasets like AIM. Future research will focus on improving inference speed—as patch-based processing is computationally intensive—and developing more diverse, ultra-high-resolution training sets to further refine the model's ability to capture the finest possible details.
Summary updated