Toward High-quality Face-Mask Occluded Restoration

Feihong Lu, Hang Chen, Kang Li, Qiliang Deng, Jian Zhao, Kaipeng Zhang, Hong Han

Abstract

Face-mask occluded restoration aims at restoring the masked region of a human face, which has attracted increasing attention in the context of the COVID-19 pandemic. One major challenge of this task is the large visual variance of masks in the real world. To solve it we first construct a large-scale Face-mask Occluded Restoration (FMOR) dataset, which contains 5,500 unmasked images and 5,500 face-mask occluded images with various illuminations, and involves 1,100 subjects of different races, face orientations, and mask types. Moreover, we propose a Face-Mask Occluded Detection and Restoration (FMODR) framework, which can detect face-mask regions with large visual variations and restore them to realistic human faces. In particular, our FMODR contains a self-adaptive contextual attention module specifically designed for this task, which is able to exploit the contextual information and correlations of adjacent pixels for achieving high realism of the restored faces, which are however often neglected in existing contextual attention models. Our framework achieves state-of-the-art results of face restoration on three datasets, including CelebA, AR, and our FMOR datasets. Moreover, experimental results on AR and FMOR datasets demonstrate that our framework can significantly improve masked face recognition and verification performance.
BibTeX
@article{lu2023toward,
  author = {Feihong Lu and Hang Chen and Kang Li and Qiliang Deng and Jian Zhao and Kaipeng Zhang and Hong Han},
  title = {{Toward High-quality Face-Mask Occluded Restoration}},
  journal = {ACM Trans. Multim. Comput. Commun. Appl.},
  year = {2023},
  doi = {10.1145/3524137},
}

Loading related content...

Figure