PrimateFace: A Machine Learning Resource for Automated Face Analysis in Human and Non-human Primates

Felipe Parodi, Jordan Matelsky, Ap Lamacchia, Melanie Segado, Yaoguang Jiang, Alejandra Regla-Vargas, L Sofi, Clare M. Kimock, Bridget M. Waller, M. L Platt, Konrad P. KördingbioRxiv 2025 · DOI

Abstract

Machine learning has revolutionized human face analysis, but equivalent tools for non-human primates remain limited and species-specific, hindering progress in neuroscience, anthropology, and conservation. Here, we present PrimateFace, a comprehensive, cross-species platform for primate facial analysis comprising a systematically curated dataset of 260,000+ images spanning over 60 genera, including a genus-balanced subset of 60,000 images, annotated with bounding boxes and facial landmark configurations. Face detection and facial landmark estimation models trained on PrimateFace achieve high cross-species performance, from tarsiers to gorillas, achieving performance comparable to baseline models trained exclusively on human data (0.34 vs. 0.39 mAP for face detection; 0.061 vs. 0.053 normalized landmark error), demonstrating the generalization benefits of cross-species training. PrimateFace enables diverse downstream applications including individual recognition, gaze analysis, and automated extraction of stereotyped (e.g., lip-smacking) and subtle (e.g., soft left turn) facial movements. PrimateFace provides a standardized platform for facial phenotyping across the primate order, empowering data-driven studies that advance the health and well-being of human and non-human primates.

Introduction

Facial analysis is a cornerstone of modern computer vision, yet its benefits have largely been confined to a single species: Homo sapiens. For researchers studying our closest relatives, the lack of robust, cross-species tools has meant falling back on labor-intensive manual coding or species-specific models that fail to generalize. PrimateFace introduces a massive open-source ecosystem designed to change that, providing the data and models needed to automate facial analysis across the entire primate order.

The Challenge of Primate Facial Analysis

While human facial recognition and landmark tracking are highly mature, non-human primate (NHP) facial analysis faces three major hurdles:

  1. Morphological Diversity: The vast differences in fur, skin color, and snout shapes (from the flat faces of apes to the long muzzles of baboons) make human-trained models perform poorly on NHPs.
  2. Annotation Costs: Manually labeling landmarks on thousands of images across dozens of species is prohibitively expensive.
  3. Data Scarcity: Most existing NHP datasets are small and species-specific, hindering the development of "foundation models" for primatology.

PrimateFace tackles these by creating a unified, taxonomically balanced resource that moves beyond species-specific solutions toward a "species-agnostic" approach.

The PrimateFace Ecosystem

The authors developed an integrated ecosystem that covers the entire pipeline from data curation to scientific application. This workflow integrates curation from open-source datasets and web crawls with a sophisticated annotation pipeline and a diverse model family.

The PrimateFace integrated ecosystem for dataset curation, model development, and scientific application.

Methodology and Dataset Curation

Key technical highlights of the PrimateFace methodology include:

  • DINOv2-Guided Sampling: To maximize visual and taxonomic diversity without labeling every image, the team used DINOv2 (a self-supervised vision transformer) to embed images and then used "Farthest Point Sampling" to select the most representative 10,000 images for initial manual annotation.
  • Hybrid Annotation Workflow: An iterative "human-in-the-loop" process was employed where models were trained on small batches, generated "pseudo-labels" for new images, and were then corrected by humans in a custom GUI.
  • Landmark Conversion Model: To bridge different research standards, an attention-based MLP was built to convert between the custom 48-point NHP-centric schema and the human 68-point standard.

The resulting PrimateFace dataset includes over 260,000 total images, with a 60,000-image genus-balanced subset ensuring that rare taxa like tarsiers or lemurs are well-represented alongside common ones like macaques.

Performance and Cross-Species Generalization

One of the most striking findings is that while human-trained models fail on primates, primate-trained models work remarkably well on humans. The morphological diversity of the primate order acts as a form of "natural data augmentation," creating more robust feature representations than human-only data.

Performance metrics including face detection by superfamily, domain transfer between PrimateFace and WIDERFace, and image sampling efficiency.

As shown in the results, PrimateFace models achieved 0.34 mAP on the challenging WIDERFace human benchmark, nearly matching models trained exclusively on human data (0.39 mAP). Conversely, human models showed dramatic performance degradation when applied to primates. Furthermore, in low-data regimes (e.g., only 500 images), DINOv2-guided sampling significantly outperformed random sampling, proving that the selection of representative images is critical for model performance.

Scientific Utility and Applications

PrimateFace enables a wide range of real-world scientific applications, demonstrating its utility beyond simple detection:

  • Individual Recognition: Training a macaque identification system with 86% accuracy in under an hour.
  • Vocal-Motor Coupling: Correlating mouth aperture with acoustic signals in howling monkeys.
  • Behavioral Discovery: Using unsupervised learning to identify "behavioral syllables" like lip-smacking that differ between male and female macaques.
  • Gaze Estimation: Extracting gaze angles and cross-individual gaze influence.

Conclusion and Future Work

PrimateFace provides a standardized platform for facial phenotyping across the primate order. While the current models are robust, future work will aim to expand the dataset to include more diverse behaviors, integrate 3D facial structure, and develop automated detection for "Action Units" across different species. All models, notebooks, and the dataset are available at github.com/KordingLab/PrimateFace.

Summary updated

Abstract

Abstract Machine learning has revolutionized human face analysis, but equivalent tools for non-human primates remain limited and species-specific, hindering progress in neuroscience, anthropology, and conservation. Here, we present PrimateFace, a comprehensive, cross-species platform for primate facial analysis comprising a systematically curated dataset of 260,000+ images spanning over 60 genera, including a genus-balanced subset of 60,000 images, annotated with bounding boxes and facial landmark configurations. Face detection and facial landmark estimation models trained on PrimateFace achieve high cross-species performance, from tarsiers to gorillas, achieving performance comparable to baseline models trained exclusively on human data (0.34 vs. 0.39 mAP for face detection; 0.061 vs. 0.053 normalized landmark error), demonstrating the generalization benefits of cross-species training. PrimateFace enables diverse downstream applications including individual recognition, gaze analysis, and automated extraction of stereotyped (e.g., lip-smacking) and subtle (e.g., soft left turn) facial movements. PrimateFace provides a standardized platform for facial phenotyping across the primate order, empowering data-driven studies that advance the health and well-being of human and non-human primates. All models, notebooks, and data can be found at github.com/KordingLab/PrimateFace .
BibTeX
@misc{parodi2025primateface,
  author = {Felipe Parodi and Jordan Matelsky and AP Lamacchia and Melanie Segado and Yaoguang Jiang and Alejandra Regla-Vargas and L Sofi and Clare M. Kimock and Bridget M. Waller and M. L Platt and Konrad P. Körding},
  title = {{PrimateFace: A Machine Learning Resource for Automated Face Analysis in Human and Non-human Primates}},
  year = {2025},
  doi = {10.1101/2025.08.12.669927},
  howpublished = {bioRxiv},
}
Figure