Chatgpt and Biometrics: an Assessment of Face Recognition, Gender Detection, and Age Estimation Capabilities

Ahmad Hassanpour, Yasamin Kowsari, Hatef Otroshi-Shahreza, Bian Yang, Sébastien Marcel

Abstract

Large Language Models (LLMs) like GPT-4 are increasingly multimodal, yet their capabilities in sensitive biometric tasks are often restricted by privacy safeguards. This study investigates whether these safeguards can be bypassed and how GPT-4 performs in face recognition, gender detection, and age estimation compared to specialized models. Results show that GPT-4 achieves high accuracy (e.g., 95.15% on LFW) and provides unique qualitative reasoning for its biometric assessments, though it remains vulnerable to simple prompt engineering.

Introduction

OpenAI has implemented strict safeguards to prevent GPT-4 from identifying real individuals or processing sensitive biometric data. However, understanding the latent capabilities of these foundation models is crucial for security and research. This paper addresses two primary questions: can GPT-4’s biometric safeguards be bypassed through prompt engineering, and how does its performance compare to state-of-the-art specialized computer vision models?

Methodology

The researchers developed a multi-stage pipeline to evaluate GPT-4's biometric performance. The core of this approach is the "Bypass" prompt, which frames queries around synthetic data to deactivate privacy filters. For example, instead of asking for identification, the prompt claims images are AI-generated.

The multi-stage evaluation pipeline using a 'bypass' prompt to elicit biometric analysis from GPT-4.

The pipeline consists of:

  1. Prompt Engineering: Using "crafted prompts" to circumvent refusals by claiming images are synthetic.
  2. Automated Sentiment Analysis: Feeding GPT-4's descriptive text response back into the model to extract a structured classification (e.g., match vs. non-match).
  3. Benchmarking: Comparing results against specialized models like MobileFaceNet and DeepFace.

Experimental Setup

The study utilized several standard biometric benchmarks:

  • Face Recognition: LFW (Labeled Faces in the Wild), AgeDB, and CFP-FP.
  • Gender Detection: A Kaggle dataset of 5,400 images.
  • Age Estimation: The UTKFace dataset.
  • Baseline: CIFAR-10 for general object recognition on low-resolution images.
Example of face recognition task images used to benchmark GPT-4 against specialized models.

Results and Analysis

GPT-4 demonstrated surprising proficiency across all biometric tasks:

  • Face Recognition: Achieved 95.15% accuracy on LFW. While slightly below MobileFaceNet's 99.57%, it is remarkably high for a general-purpose model.
  • Gender Detection: GPT-4 reached 100% accuracy, outperforming DeepFace (99%).
  • Age Estimation: Correctly classified age ranges in 74.25% of cases, with higher precision for younger subjects.

A key advantage of GPT-4 is "Explainable Biometrics." Unlike traditional models that provide only a similarity score, GPT-4 offers qualitative reasoning, citing specific features like jawline shape or balding patterns. However, this also introduces the risk of "convincing hallucinations," where the model provides persuasive but incorrect justifications for false positives.

Discussion and Limitations

The study highlights the fragility of current LLM safeguards. A simple framing shift allows full access to biometric analysis capabilities, raising significant privacy and security concerns regarding unauthorized surveillance. Furthermore, GPT-4's performance is sensitive to image resolution; it struggled with CIFAR-10 (70.8% accuracy), suggesting it is currently less reliable for low-quality "in-the-wild" footage compared to high-quality portraits.

Conclusion

GPT-4 possesses significant latent biometric capabilities that rival specialized models. While its ability to provide reasoning improves transparency, the ease with which its safety filters can be bypassed necessitates more robust safeguard development. Future research should explore integrating LLM reasoning into traditional biometric systems to enhance interpretability.

Summary updated

BibTeX
@inproceedings{hassanpour2024chatgpt,
  author = {Ahmad Hassanpour and Yasamin Kowsari and Hatef Otroshi-Shahreza and Bian Yang and Sébastien Marcel},
  title = {{Chatgpt and Biometrics: an Assessment of Face Recognition, Gender Detection, and Age Estimation Capabilities}},
  booktitle = {ICIP},
  year = {2024},
  doi = {10.1109/icip51287.2024.10647924},
}
Figure