Abstract
Artificially created human faces play an increasingly important role in our digital world. However, the so-called uncanny valley effect may cause people to perceive highly, yet not perfectly human-like faces as eerie, bringing challenges to the interaction with virtual agents. At the same time, the neurocognitive underpinnings of the uncanny valley effect remain elusive. Here, we utilized an electroencephalography (EEG) dataset of steady-state visual evoked potentials (SSVEP) in which participants were presented with human face images of different stylization levels ranging from simplistic cartoons to actual photographs. Assessing neuronal responses both in frequency and time domain, we found a non-linear relationship between SSVEP amplitudes and stylization level, that is, the most stylized cartoon images and the real photographs evoked stronger responses than images with medium stylization. Moreover, realness of even highly similar stylization levels could be decoded from the EEG data with task-related component analysis (TRCA). Importantly, we also account for confounding factors, such as the size of the stimulus face's eyes, which previously have not been adequately addressed. Together, this study provides a basis for future research and neuronal benchmarking of real-time detection of face realness regarding three aspects: SSVEP-based neural markers, efficient classification methods, and low-level stimulus confounders.
Introduction
As virtual humans become ubiquitous in gaming, film, and AI-driven communication, understanding how our brains perceive "realness" is critical. A major hurdle in computer graphics is the Uncanny Valley (UV): the phenomenon where a near-perfect human replica triggers a sense of unease or "eeriness" in observers. While the UV effect is well-documented through subjective surveys, these ratings are prone to bias and variability.
The neurocognitive underpinnings—what is actually happening in the brain—remain elusive. Furthermore, previous studies often failed to account for "low-level" visual confounders. For example, do our brains react to the "fakeness" of a cartoon, or simply to the fact that cartoon characters often have disproportionately large eyes? This paper seeks to find objective neural markers for face realness while controlling for these confounding factors.
Experimental Design and Stimuli
The study re-analyzed an EEG dataset originally presented by Bagdasarian et al. (2020), involving 10 subjects. The researchers utilized Steady-State Visual Evoked Potentials (SSVEP), where participants were shown face images flickering at a frequency of 5 Hz. This periodic stimulation drives a robust, synchronized neural response at the same frequency.
The stimuli consisted of 36 face images across six stylization levels (R0 to R5). R0 represents simplistic, abstract cartoons, while R5 represents authentic photographs. The set included variations in gender and three emotional expressions: neutral, happy, and angry. To maximize the signal-to-noise ratio of the 5 Hz oscillations, the researchers applied Spatio-Spectral Decomposition (SSD) to the EEG data.
Results: The Neural "Valley"
The most significant finding was the non-linear relationship between stylization and brain activity. Both the most abstract cartoons (R0) and the real photos (R5) evoked stronger neural responses (in both SSVEP and the N170 ERP component) than the medium-stylized images in the middle. This "U-shape" provides a biological mirror to the behavioral Uncanny Valley.
Statistical modeling using Linear Mixed-Effect Models (LMM) confirmed that quadratic models significantly outperformed linear models for both SSVEP amplitudes ($p < 0.001$) and N170 components ($p < 0.001$). Importantly, this quadratic neural effect persisted even after statistically controlling for eye size and luminosity, suggesting the brain processes "realness" as a high-level construct rather than a reaction to simple geometry.
Decoding Realness with TRCA
To test the feasibility of real-time detection, the team employed Task-Related Component Analysis (TRCA), a technique typically used in high-speed Brain-Computer Interfaces (BCIs). TRCA extracts task-related components by maximizing the covariance between EEG segments across different sessions.
- Six-Class Accuracy: Distinguishing all levels (R0-R5) achieved 47.46% accuracy using 8 seconds of data (chance is 16.7%). Even with just 2 seconds of data, accuracy remained high at 39.48%.
- Binary Classification: Distinguishing the "easiest" pair (R0 vs. R5) reached ~86%, while the "hardest" pair (R4 vs. R5) reached ~66%.
The ability to distinguish between R4 (highly realistic computer-generated faces) and R5 (real photos) suggests that EEG can detect "fakeness" even when human observers might struggle to articulate the difference.
Conclusion and Future Directions
While limited by a small sample size (10 participants), this study provides a foundation for neuronal benchmarking of face realness. The authors suggest that this SSVEP-based decoding could eventually be used in real-time systems for computer graphics design. A designer wearing an EEG cap could receive real-time feedback on a character's "realness," allowing them to pinpoint exactly when they have successfully crossed the Uncanny Valley. Future work should "zoom in" on the specific transition points of the valley with more granular stimuli to identify the exact threshold of the neural dip.
Summary updated