Indian Masked Faces in the Wild Dataset

Shiksha Mishra, Puspita Majumdar, Richa Singh, Mayank Vatsa

Abstract

Due to the COVID-19 pandemic, wearing face masks has become a mandate in public places worldwide. Face masks occlude a significant portion of the facial region. Additionally, people wear different types of masks, from simple ones to ones with graphics and prints. These pose new challenges to face recognition algorithms. Researchers have recently proposed a few masked face datasets for designing algorithms to overcome the challenges of masked face recognition. However, existing datasets lack the cultural diversity and collection in the unrestricted settings. Country like India with attire diversity, people are not limited to wearing traditional masks but also clothing like a thin cotton printed towel (locally called as "gamcha"), "stoles", and "handkerchiefs" to cover their faces. In this paper, we present a novel Indian Masked Faces in the Wild (IMFW) dataset which contains images with variations in pose, illumination, resolution, and the variety of masks worn by the subjects. We have also benchmarked the performance of existing face recognition models on the proposed IMFW dataset. Experimental results demonstrate the limitations of existing algorithms in presence of diverse conditions.

Introduction

Face recognition technology has reached high levels of maturity, but the COVID-19 pandemic introduced a massive "occlusion" problem: masks. While several masked face datasets emerged quickly, they often suffered from demographic bias—focusing primarily on Caucasian or North Asian populations—and assumed a level of mask uniformity (standard surgical or N95 masks) that does not reflect global reality.

In the Indian context, face coverings are exceptionally diverse. Beyond medical masks, individuals frequently utilize traditional attire such as gamchas (cotton towels), stoles, or handkerchiefs. This cultural variation, combined with unconstrained "in-the-wild" environments, creates unique patterns of occlusion that existing algorithms are ill-equipped to handle.

The IMFW Dataset

To address these gaps, we introduce the Indian Masked Faces in the Wild (IMFW) dataset. The dataset comprises 1,374 images (630 masked, 744 non-masked) across 200 subjects, sourced to ensure a representative mix of conditions:

  • Indian Celebrity Set: Web-scraped images of actors, athletes, and politicians.
  • Instagram Set: Public profile images featuring high variation in lighting and backgrounds.
  • Indian Crowd Set: Volunteer-contributed images captured via mobile devices (48MP/13MP) to simulate real-world smartphone recognition scenarios.
Figure 1: Examples of diverse face coverings in the IMFW dataset, including gamchas, stoles, and printed masks.

Technical Challenges and Insights

The primary difficulty in masked face recognition lies in the increased intra-class variation. As shown in the dataset, the same individual can appear drastically different depending on whether they are wearing a standard mask, a patterned cloth, or a gamcha.

Figure 6: Illustration of intra-class variation (same subject with different masks) and inter-class variation (different subjects) within the IMFW dataset.

A particularly novel challenge identified is the "Printed Face" problem. Masks featuring facial graphics can deceive detection algorithms, causing them to focus on the mask's print rather than the subject's actual features, or confusing the features of the print with the subject's periocular region. Furthermore, traditional cloths like gamchas do not follow the standard geometry of medical masks; they wrap around the head and neck in ways that synthetic mask generators fail to model accurately.

Benchmarking and Results

We evaluated several state-of-the-art pre-trained models, including VGGFace, ResNet50, LightCNN29, and ArcFace. The results highlight a significant performance gap when moving from standard datasets to the IMFW benchmark. For instance, models achieving ~96% accuracy on standard benchmarks like LFWA saw performance drop to approximately 54% on IMFW.

ArcFace emerged as the strongest baseline, achieving a 57.96% Rank-1 identification accuracy. To improve performance, we fine-tuned LightCNN29 using the IMFW training split and Triplet Loss, which successfully boosted Rank-1 accuracy from 53.97% to 66.47%. Interestingly, models performed best on the "Crowd" subset, likely because the mobile-captured images lacked the extreme pose and resolution variations found in the web-scraped celebrity and Instagram sets.

Conclusion and Future Work

The IMFW dataset demonstrates that masked face recognition in culturally diverse and unconstrained settings remains an open challenge. While fine-tuning offers improvements, current models still struggle with the complex occlusions presented by traditional Indian attire. Future research will focus on scaling the dataset, developing architectures that prioritize the periocular region, and creating more sophisticated synthetic augmentation techniques that better reflect global diversity in face coverings.

Summary updated

Abstract

Due to the COVID-19 pandemic, wearing face masks has become a mandate in public places worldwide. Face masks occlude a significant portion of the facial region. Additionally, people wear different types of masks, from simple ones to ones with graphics and prints. These pose new challenges to face recognition algorithms. Researchers have recently proposed a few masked face datasets for designing algorithms to overcome the challenges of masked face recognition. However, existing datasets lack the cultural diversity and collection in the unrestricted settings. Country like India with attire diversity, people are not limited to wearing traditional masks but also clothing like a thin cotton printed towel (locally called as ``gamcha''), ``stoles'', and ``handkerchiefs'' to cover their faces. In this paper, we present a novel \textbf{Indian Masked Faces in the Wild (IMFW)} dataset which contains images with variations in pose, illumination, resolution, and the variety of masks worn by the subjects. We have also benchmarked the performance of existing face recognition models on the proposed IMFW dataset. Experimental results demonstrate the limitations of existing algorithms in presence of diverse conditions.
BibTeX
@inproceedings{mishra2021indian,
  author = {Shiksha Mishra and Puspita Majumdar and Richa Singh and Mayank Vatsa},
  title = {{Indian Masked Faces in the Wild Dataset}},
  booktitle = {ICIP},
  year = {2021},
  url = {https://openreview.net/forum?id=KSWoRw96lR1L},
}
Figure