Processing of Individual Items during Ensemble Coding of Facial Expressions

Huiyun Li, Luyan Ji, Ke Tong, Naixin Ren, Wenfeng Chen, Chang Hong Liu, Xiaolan FuFrontiers in Psychology 2016 · DOI

Abstract

There is growing evidence that human observers are able to extract the mean emotion or other type of information from a set of faces. The most intriguing aspect of this phenomenon is that observers often fail to identify or form a representation for individual faces in a face set. However, most of these results were based on judgments under limited processing resource. We examined a wider range of exposure time and observed how the relationship between the extraction of a mean and representation of individual facial expressions would change. The results showed that with an exposure time of 50 ms for the faces, observers were more sensitive to mean representation over individual representation, replicating the typical findings in the literature. With longer exposure time, however, observers were able to extract both individual and mean representation more accurately. Furthermore, diffusion model analysis revealed that the mean representation is also more prone to suffer from the noise accumulated in redundant processing time and leads to a more conservative decision bias, whereas individual representations seem more resistant to this noise. Results suggest that the encoding of emotional information from multiple faces may take two forms: single face processing and crowd face processing.

Introduction

Human visual environments are often visually overwhelming. To manage this complexity, our visual system employs "ensemble coding"—a compression technique where the brain extracts summary statistics, such as average emotion or size, from a group of objects. A long-standing debate in psychology is whether we calculate this "mean" only after processing individual items or if the mean is an "element-independent" shortcut that bypasses detailed individual analysis.

Previous research has often shown that people can identify the average emotion of a set of faces even when they fail to recognize the individual faces within that set. However, much of this work relied on very brief exposure times. This paper investigates whether the relationship between individual and ensemble coding changes when the brain is given more time to process the scene.

Methods and Materials

The researchers conducted two main experiments using a controlled stimulus set of 50 grayscale images. These images were created by morphing between a neutral expression and a disgusted expression of the same female model, with each step representing approximately a 2% change in emotion.

Example of a disgusted facial expression used in the morphing sequence.

Experimental Design

  1. Experiment 1 (Membership Identification): Participants viewed a set of four faces for varying durations (50ms to 2000ms). They were then asked if a test face—which could be an actual member, the mathematical mean of the set, or a distractor—was part of the original group.
  2. Experiment 2 (Mean Discrimination): Participants were asked to explicitly compare the average emotion of a four-face set to a previously shown reference face, testing the precision of explicit ensemble coding.

To analyze the cognitive components of these tasks, the authors applied a Hierarchical Diffusion Model (HDM). This tool decomposes reaction times and accuracy into the speed of information accumulation (drift rate, $v$), the decision threshold (boundary separation, $a$), and non-decisional encoding time ($t_0$).

Key Insights and Results

The 50ms Shortcut vs. The 500ms Turning Point

At extremely brief exposures (50ms), participants were significantly more likely to identify the "mean" face as a member than the actual faces they had just seen. This supports the idea that ensemble coding is a rapid, "global-first" process. However, as exposure time increased to 500ms, the representation of individual faces became significantly more precise. The brain effectively moves from a "coarse sketch" to a detailed "high-resolution" map of individuals.

The Noise of Redundancy

Surprisingly, performance in the mean discrimination task eventually suffered with too much time. Diffusion modeling showed that the drift rate ($v$) for the mean followed an inverse U-curve, peaking at 500ms and then declining. This suggests that the mean representation is prone to noise accumulated during redundant processing time. Furthermore, the boundary separation ($a$) increased over time, indicating that as the scene became more complex, participants became more conservative and cautious in their decision-making.

Conclusion

The results suggest that the encoding of emotional information from multiple faces may take two distinct forms: crowd processing and single face processing. Crowd processing is fast and efficient but susceptible to noise over time. Single face processing is slower to build but more resistant to the noise accumulated during long exposures.

While these findings clarify the temporal dynamics of social perception, the study's use of a single model's face limits its ecological validity. In real-world crowds, differences in identity, age, and gender likely add significant noise to the ensemble coding process. Future research should investigate how the brain weights individual faces when calculating a crowd's emotional "gist."

Summary updated

Abstract

There is growing evidence that human observers are able to extract the mean emotion or other type of information from a set of faces. The most intriguing aspect of this phenomenon is that observers often fail to identify or form a representation for individual faces in a face set. However, most of these results were based on judgments under limited processing resource. We examined a wider range of exposure time and observed how the relationship between the extraction of a mean and representation of individual facial expressions would change. The results showed that with an exposure time of 50 ms for the faces, observers were more sensitive to mean representation over individual representation, replicating the typical findings in the literature. With longer exposure time, however, observers were able to extract both individual and mean representation more accurately. Furthermore, diffusion model analysis revealed that the mean representation is also more prone to suffer from the noise accumulated in redundant processing time and leads to a more conservative decision bias, whereas individual representations seem more resistant to this noise. Results suggest that the encoding of emotional information from multiple faces may take two forms: single face processing and crowd face processing.
BibTeX
@inproceedings{li2016processing,
  author = {Huiyun Li and Luyan Ji and Ke Tong and Naixin Ren and Wenfeng Chen and Chang Hong Liu and Xiaolan Fu},
  title = {{Processing of Individual Items during Ensemble Coding of Facial Expressions}},
  booktitle = {Frontiers in Psychology},
  year = {2016},
  doi = {10.3389/fpsyg.2016.01332},
}
Figure