AuthNet: A Deep Learning Based Authentication Mechanism Using Temporal Facial Feature Movements (Student Abstract)

Mohit Raghavendra, Pravan Omprakash, B. R. MukeshAAAI 2021 · DOI

Abstract

AuthNet is a language-agnostic authentication system that combines facial recognition with the temporal dynamics of facial movements during speech. By analyzing the unique way a person’s face moves while uttering a password, rather than just static imagery, it provides a robust defense against spoofing attacks. The system utilizes a hybrid CNN-RNN architecture to achieve high accuracy across diverse datasets and languages, offering a contact-less and secure biometric solution.

Introduction

Standard facial recognition systems are increasingly vulnerable to "presentation attacks," where imposters use high-resolution photos or videos to gain unauthorized access. While visual speech recognition (lip-reading) has been proposed as a countermeasure, existing models often struggle with language dependency, lighting variations, and noisy environments. AuthNet addresses these challenges by shifting the focus from simple "face matching" to "behavioral matching," capturing the unique temporal signatures of facial feature transitions during speech.

Methodology

AuthNet employs a two-level hybrid deep learning architecture to process video sequences:

  1. Spatial Feature Extraction: Video frames are processed using a Haar cascade face detector. The cropped face regions are fed into a pre-trained VGGFace model to extract spatial feature vectors representing facial characteristics.
  2. Temporal Modeling: These vectors are passed through a network of four LSTM (Long Short-Term Memory) layers, each consisting of 20 timesteps. This allows the model to learn the specific rhythm and muscle movements unique to an individual saying a specific word.
  3. Classification: A final sigmoid activation layer predicts the probability of a match for the specific "person-word" combination.

Experimental Setup

The model was evaluated on two distinct datasets:

  • MIRACL-VC1: A standard public dataset for visual speech recognition used for benchmarking against state-of-the-art methods.
  • Custom Dataset: A manually compiled dataset of 35 "person-word" combinations (5 speakers, 7 words) captured via smartphones in English and Hindi under varying lighting conditions to test real-world robustness.

Analysis of Feature Variance

A critical insight of this research is the distinctiveness of temporal facial signatures. As shown in Figure 1, there is a significant difference in feature vectors when comparing different individuals uttering the same word, or the same individual uttering different words. This variance confirms that the temporal movement of facial features serves as a viable and unique biometric signature, allowing the system to distinguish between users even when the spoken content is identical.

Feature vector differences across various authentication scenarios.

Results and Discussion

AuthNet demonstrated superior performance compared to existing hybrid models. On the MIRACL-VC1 dataset, it achieved an accuracy of 98.1%, outperforming other architectures like FaceNet+LipNet.

Comparison of AuthNet accuracy against state-of-the-art models.

Key performance metrics include:

  • Robustness: Maintained a high accuracy of 98.0% on the custom smartphone dataset.
  • Security: Successfully rejected 98% of "different person" attacks and 94% of "wrong password" attempts by the authorized user.
  • Efficiency: High performance was achieved with as few as 10 positive training samples per user, suggesting viability for quick enrollment.

Conclusion

AuthNet provides a language-agnostic, dynamic authentication mechanism that is resilient to common spoofing techniques. By focusing on the visual utterance rather than phonetic data, it offers a global solution for secure access. Future research will focus on scaling the system to larger populations and benchmarking computational overhead for mobile deployment.

Summary updated

BibTeX
@inproceedings{raghavendra2021authnet,
  author = {Mohit Raghavendra and Pravan Omprakash and B. R. Mukesh},
  title = {{AuthNet: A Deep Learning Based Authentication Mechanism Using Temporal Facial Feature Movements (Student Abstract)}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i18.17933},
}
Figure