Subclass Contrastive Loss for Injured Face Recognition

Puspita Majumdar, Saheb Chhabra, Richa Singh, Mayank Vatsa

TL;DR

This paper introduces the first dedicated framework for recognizing faces with severe injuries, such as those sustained in accidents or disasters. The authors propose a novel Subclass Contrastive Loss (SCL) and the Injured Face (IF) database to help deep learning models bridge the gap between a person's "normal" appearance and their injured state.

What Problem Does This Paper Address?

Standard face recognition (FR) systems are incredibly accurate today, but they are typically trained on "clean" datasets. When a face is distorted by swelling, bruises, blood clots, or lacerations, the underlying geometry and texture change significantly.

In forensic and emergency scenarios—like identifying unconscious accident victims or deceased individuals—this "domain gap" causes existing FR models to fail. Currently, responders must rely on slow, expensive methods like DNA or dental profiling. This paper aims to make automated face recognition a viable tool for these high-stakes situations by specifically accounting for the appearance changes caused by physical trauma.

Methods

The core contribution is the Subclass Contrastive Loss (SCL). While standard contrastive loss tries to pull images of the same person together and push different people apart, SCL goes a step further by treating "injured" and "non-injured" states as distinct subclasses of a single identity.

  • Intra-class Minimization: The loss function specifically minimizes the distance between the non-injured subclass and the injured subclass of the same person. It also minimizes the distance between different types of injured images of that same person.
  • Inter-class Maximization: It maximizes the distance between the subclasses of different subjects.
  • Architecture: The authors used LightCNN-9 as their base model, implementing a Siamese-style network where weights are shared between two branches during training to learn a robust embedding space.

Datasets

The authors noted a total lack of public data for this problem, so they created their own:

  • Injured Face (IF) Database: A new dataset containing 898 images of 100 subjects (346 injured and 552 non-injured). Images were sourced from the internet, covering injuries from sports, accidents, and violence.
  • SCface: Used as an "Extended Gallery" to test how the model performs when the target "injured" face must be found within a much larger pool of 100 additional "distractor" identities.

Key Insights and Ideas

  • Injuries as Subclasses: The most important "aha moment" is the realization that an injury isn't just noise; it's a systematic transformation of the face. By explicitly modeling "injured" and "normal" as subclasses, the network learns to focus on features that remain invariant even under trauma.
  • Existing Models Fail Hard: The paper demonstrates that popular pre-trained models like VGG-Face and OpenFace perform very poorly on injured faces, with Rank-1 accuracies often falling below 10%.
  • Distribution Overlap: Qualitative analysis of score distributions showed that standard losses result in a high overlap between "genuine" and "imposter" matches. SCL successfully reduces this overlap, making the system more reliable for verification.

Key Results

The proposed SCL significantly outperformed standard training objectives on the IF database:

  • Rank-1 Accuracy: SCL achieved 36.70%, compared to 28.67% for standard Contrastive Loss and 30.26% for Triplet Loss.
  • Comparison to Pre-trained Models: The best-performing pre-trained model (LightCNN-9) only reached 25.43% Rank-1 accuracy before being fine-tuned with the proposed SCL.
  • Robustness: In the "Extended Gallery" experiment (adding 100 extra identities to the search pool), the accuracy remained stable at 36.13%, suggesting the model is robust to larger search spaces.

Limitations and Future Work

  • Dataset Size: With only 100 subjects and 898 images, the IF database is small by modern deep learning standards. Larger, more diverse datasets are needed to improve generalization.
  • Absolute Accuracy: While a ~37% Rank-1 accuracy is a major improvement over the baseline, it is still quite low for real-world deployment. Identifying injured faces remains an "arduous task."
  • Injury Diversity: The paper treats all injuries under the "injured" subclass. Future work could potentially categorize injuries (e.g., "swelling" vs. "laceration") to further refine the loss function.
  • Ethical Considerations: The authors acknowledge the sensitivity of using images of injured or deceased individuals, noting that such images are often not suitable for public media but are necessary for forensic identification.

Summary updated

Abstract

Deaths and injuries are common in road accidents, violence, and natural disaster. In such cases, one of the main tasks of responders is to retrieve the identity of the victims to reunite families and ensure proper identification of deceased/injured individuals. Apart from this, identification of unidentified dead bodies due to violence and accidents is crucial for the police investigation. In the absence of identification cards, current practices for this task include DNA profiling and dental profiling. Face is one of the most commonly used and widely accepted biometric modalities for recognition. However, face recognition is challenging in the presence of facial injuries such as swelling, bruises, blood clots, laceration, and avulsion which affect the features used in recognition. In this paper, for the first time, we address the problem of injured face recognition and propose a novel Subclass Contrastive Loss (SCL) for this task. A novel database, termed as Injured Face (IF) database, is also created to instigate research in this direction. Experimental analysis shows that the proposed loss function surpasses existing algorithm for injured face recognition.
BibTeX
@inproceedings{majumdar2019subclass,
  author = {Puspita Majumdar and Saheb Chhabra and Richa Singh and Mayank Vatsa},
  title = {{Subclass Contrastive Loss for Injured Face Recognition}},
  booktitle = {BTAS},
  year = {2019},
  url = {https://openreview.net/forum?id=0h6taEGuCH2},
}

Loading related content...

Figure