Inductive Guided Filter: Real-Time Deep Matting with Weakly Annotated Masks on Mobile Devices

Yaoyi Li, Jianfu Zhang, Weijie Zhao, Weihao Jiang, Hongtao Lu

Abstract

Image matting is a critical task for high-quality photo and video editing, yet it has traditionally required computationally expensive models and precise, hand-drawn "trimaps." We present the Inductive Guided Filter (IGF), a lightweight framework designed for real-time performance on mobile devices. By parameterizing the classical Guided Filter within a deep learning architecture, IGF achieves state-of-the-art results using only "weakly annotated" (noisy) masks. Our method runs at over 30 FPS on mobile hardware and significantly outperforms existing models like Adobe DIM in robustness to input noise.

Introduction: The Challenge of Mobile Matting

Separating a foreground object from its background with per-pixel transparency (the alpha matte, $\alpha$) is a complex task, especially for fine details like hair, fur, or semi-transparent edges. Two major obstacles have prevented its widespread use on mobile: high computational cost and the need for precise user-provided trimaps. Existing high-quality methods are too slow for mobile hardware and often fail when provided with the rough, noisy masks typically generated by quick mobile segmentation tools or finger-traces.

Methods: The Inductive Guided Filter

The core innovation of this work is the Inductive Guided Filter (IGF). Instead of treating the Guided Filter as a per-image optimization problem, the authors use a neural network to learn the filter's parameters.

The IGF architecture uses a lightweight Hourglass backbone with spatial attention to generate coefficient maps A and B, which are then used to produce the final alpha matte.

As shown in the architecture above, a tiny Hourglass (U-Net style) backbone, optimized with depthwise convolutions, processes the input image $I$ and a weak mask $M$. A spatial attention mechanism focuses the model on difficult boundary regions. The network outputs two coefficient maps, $A$ and $B$, which are applied to the original image to "induce" the final alpha matte $\tilde{\alpha}$. This approach allows the model to leverage high-resolution image details to refine the noisy initial mask.

To further improve quality, the authors introduced a Gabor Loss to capture fine textures and a GAN framework with a PatchGAN discriminator to ensure visual sharpness and realism.

Key Results and Robustness

The IGF demonstrates a massive leap in both efficiency and robustness. On mobile hardware, it achieves real-time speeds: 15.7ms on an iPhone Xs and 25.9ms on an iPhone SE.

Qualitative comparison showing that IGF (Ours) produces significantly sharper and more accurate mattes than Adobe DIM or standard Guided Filters when provided with noisy, weakly annotated masks.

Quantitatively, IGF achieved a Mean Squared Error (MSE) of 0.028 on the MAT-2793 test set, a substantial improvement over Adobe DIM's 0.148 under the same noisy-mask conditions. As seen in the visual comparisons, the IGF successfully "snaps" to fine details like hair and tree branches even when the input mask is a rough, low-resolution blob. This robustness to noise is the "aha moment" of the paper, enabling high-quality matting from simple user inputs.

Conclusion

The Inductive Guided Filter brings professional-grade image matting to mobile devices. By learning to parameterize a robust filter, it overcomes the limitations of noisy inputs and limited compute. Future work may focus on fully "mask-free" matting and improving temporal consistency for video applications to prevent flickering in real-time streams.

Summary updated

Abstract

Recently, significant progress has been achieved in deep image matting. Most of the classical image matting methods are time-consuming and require an ideal trimap which is difficult to attain in practice. An efficient image matting method based on a weakly annotated mask is in demand for mobile applications. In this paper, we propose a novel method called Inductive Guided Filter, which tackles the real-time general image matting task with weakly annotated masks on mobile devices. The Inductive Guided Filter exploits the gradient prior implicit in Guided Filter to reduce the computational burden tremendously in a deep learning manner. The use of Gabor loss is also proposed for complicated textures in image matting. Moreover, we create an image matting dataset MAT-2793 with a variety of foreground objects. Experimental results demonstrate that our proposed method massively reduces running time with robust accuracy.
BibTeX
@inproceedings{li2020inductive,
  author = {Yaoyi Li and Jianfu Zhang and Weijie Zhao and Weihao Jiang and Hongtao Lu},
  title = {{Inductive Guided Filter: Real-Time Deep Matting with Weakly Annotated Masks on Mobile Devices}},
  booktitle = {ICME},
  year = {2020},
  doi = {10.1109/icme46284.2020.9102873},
}
Figure