How face perception unfolds over time

Katharina Dobs, Leyla Işık, Dimitrios Pantazis, Nancy Kanwisher

Abstract

Within a fraction of a second of viewing a face, we have already determined its gender, age and identity. A full understanding of this remarkable feat will require a characterization of the computational steps it entails, along with the representations extracted at each. Here, we used magnetoencephalography (MEG) to measure the time course of neural responses to faces, thereby addressing two fundamental questions about how face processing unfolds over time. First, using representational similarity analysis, we found that facial gender and age information emerged before identity information, suggesting a coarse-to-fine processing of face dimensions. Second, identity and gender representations of familiar faces were enhanced very early on, suggesting that the behavioral benefit for familiar faces results from tuning of early feed-forward processing mechanisms. These findings start to reveal the time course of face processing in humans, and provide powerful new constraints on computational theories of face perception.

Introduction

Humans possess an extraordinary ability to categorize a face’s gender, age, and identity within a fraction of a second. Despite this speed, the internal "schedule" of these computational steps has remained elusive. Does the brain process these dimensions in parallel, or is there a specific sequence? Furthermore, a well-documented behavioral advantage exists for recognizing familiar faces over strangers. Researchers have long debated whether this "familiarity effect" stems from early tuning—where visual filters are pre-calibrated for known faces—or late-stage feedback involving memory retrieval. This study tracks the temporal dynamics of neural face representations to resolve these questions.

Methodology

The researchers recorded brain activity from 16 subjects using Magnetoencephalography (MEG), providing millisecond-level precision. Subjects viewed 80 grayscale images of 16 celebrities, balanced across Familiarity (American vs. German celebrities), Gender, and Age.

To decode the high-dimensional data, the team employed:

  • Multivariate Pattern Analysis (MVPA): Support Vector Machines (SVMs) were trained to decode specific images based on MEG signals at every time point.
  • Representational Similarity Analysis (RSA): This compared "neural distances" between faces against theoretical models of gender, age, and identity.
  • CNN Controls: To isolate high-level face processing from simple visual features (e.g., hair length), the researchers "partialled out" low-level features extracted from the VGG-Face deep learning model.

Core Findings: A Coarse-to-Fine Hierarchy

The study reveals that the brain does not process all facial information simultaneously. Instead, it follows a coarse-to-fine hierarchy, resolving categorical information like gender and age before narrowing down to individual identity.

Figure 4: Time course of neural representations. Partial Spearman’s correlations between MEG data and models of gender (red), age (green), identity (blue), and familiarity (yellow). Horizontal lines indicate significant clusters.

Identity information emerged approximately 20ms later than gender and age, suggesting that distinguishing a specific individual requires more complex computational steps than broad categorization. Crucially, the neural representation of these faces at ~123ms post-stimulus correlates strongly with human behavioral perception of face similarity.

The Familiarity Advantage

A key discovery is that the enhancement for familiar faces occurs extremely early, between 70–100ms. This suggests that the "familiarity advantage" is integrated into the early, feed-forward stages of the visual system rather than being a result of slow, top-down memory retrieval. While specific identity and gender representations are boosted early, a "generic" familiarity signal (simply recognizing a face as known) appears much later, between 400–600ms.

Detailed Results and Latencies

The precise timing of these neural events provides a map of the unfolding perception:

  • Image-level decoding: Begins at 46ms.
  • Age & Gender: Emerge at 60ms and 72ms, respectively.
  • Identity: Emerges at 91ms.
Figure 5: Peak latencies for different face dimensions. Bar chart showing the time to peak representation for image, age, gender, and identity. Asterisks indicate statistical significance.

Both gender and identity representations reach their peak strength at approximately 125ms. For familiar faces, gender information was enhanced starting at 71ms and identity at 96ms, whereas unfamiliar faces showed significantly weaker identity discrimination at these early stages.

Discussion

While MEG provides exceptional temporal resolution, its spatial resolution is limited. Future research using "MEG-fMRI fusion" could pinpoint the physical brain regions corresponding to these temporal stages. Additionally, while the use of naturalistic "in the wild" images increases ecological validity, it introduces potential low-level confounds. Although controlled for via CNN models, further studies with different stimulus sets could reinforce these findings.

This research provides powerful new constraints on computational theories of face perception, demonstrating that our visual system is finely tuned to prioritize categorical information and familiar faces in the earliest stages of processing.

Summary updated

Abstract

Within a fraction of a second of viewing a face, we have already determined its gender, age and identity. A full understanding of this remarkable feat will require a characterization of the computational steps it entails, along with the representations extracted at each. Here, we used magnetoencephalography (MEG) to measure the time course of neural responses to faces, thereby addressing two fundamental questions about how face processing unfolds over time. First, using representational similarity analysis, we found that facial gender and age information emerged before identity information, suggesting a coarse-to-fine processing of face dimensions. Second, identity and gender representations of familiar faces were enhanced very early on, suggesting that the behavioral benefit for familiar faces results from tuning of early feed-forward processing mechanisms. These findings start to reveal the time course of face processing in humans, and provide powerful new constraints on computational theories of face perception.
BibTeX
@inproceedings{dobs2019how,
  author = {Katharina Dobs and Leyla Işık and Dimitrios Pantazis and Nancy Kanwisher},
  title = {{How face perception unfolds over time}},
  booktitle = {Nature Communications},
  year = {2019},
  doi = {10.1038/s41467-019-09239-1},
}
Figure