Editorial: Affective shared perception

Alessandra Sciutti, Pablo Barros, Ginevra Castellano, Yukie NagaiFrontiers in Integrative Neuroscience 2022 · DOI · OpenAlex

Introduction

Traditional robotics has long operated under the assumption that perception is a neutral, objective recording of the physical world. In this paradigm, a camera captures pixels, and an algorithm labels them with mathematical precision. However, human perception is fundamentally different: it is an inferential and subjective process. Our brains do not just record reality; they integrate sensory input with prior knowledge, social context, and internal emotional states.

This editorial introduces a collection of research focused on affective shared perception. The goal is to move robotics beyond "camera-like" sensing toward computational models that can infer, share, and adapt to the affective nuances of human interaction. By establishing a "common ground," robots can transition from being mere tools to becoming effective social partners.

A visualization of shared perception where a robot and human interact with an object, integrating physical properties with affective context.

The Challenge of Objective Perception

A significant hurdle in Human-Robot Interaction (HRI) is the "perceptual asymmetry" between the two agents. While a human perceives a social and affective reality—interpreting a gesture as an invitation or a facial expression as hesitation—the robot often misses these layers entirely. To bridge this gap, robots must achieve shared perception. This involves more than just identifying the same object; it requires understanding the history of the interaction and the partner's internal state to create a unified experience.

Multidisciplinary Methodologies

To address this complexity, the research synthesized in this editorial draws from a wide array of disciplines:

  • Neuroscience: Researchers like Manzi et al. investigate whether robots can activate the "social brain" areas in humans, testing the biological foundations of HRI.
  • Behavioral Science: Studies by Perugia et al. examine spontaneous facial mimicry toward artificial agents, while Hughson et al. explore how cultural backgrounds act as a filter for emotion recognition.
  • Computational Modeling:
    • Active Inference: Horii and Nagai utilize energy minimization frameworks to help robots autonomously select the most informative sensory modalities.
  • Bio-inspired Systems: Khan and Cañamero simulate "hormonal" mechanisms in agents, using them as functional affective states to maintain internal stability.
  • Reinforcement Learning: Churamani et al. use affect-driven frameworks to generate complex robot behaviors in social games like the Ultimatum Game.

Key Insights: From Cultural Filters to Artificial Hormones

The research highlights several counter-intuitive findings that challenge standard HRI assumptions:

  1. The "Veridical" Shift: Evidence suggests that during human-to-human interaction, the brain actually reduces its reliance on prior expectations. Instead, it shifts toward a more "veridical" (physically accurate) perception to ensure both partners are aligned on the same reality.
  2. The Anthropomorphism Paradox: Increasing a robot's human-likeness does not always improve interaction. One study found that participants mimicked the least anthropomorphic agents more frequently when they were uncertain about the emotion being expressed.
  3. Culture as a Filter: Current emotion recognition AI is often "culture-blind." Models trained on Western datasets show significant performance drops when applied to Persian or Filipino cultures, highlighting the need for culturally-aware algorithms.
  4. Hormones as Code: Modeling artificial hormonal states can serve as a powerful "social buffer," allowing agents to adapt to environmental stressors and maintain internal equilibrium in a manner similar to biological organisms.

Results and Performance

The implementation of these affective models has yielded tangible improvements in robot performance. Models utilizing active inference demonstrated higher perception accuracy even with limited sensory data. Furthermore, in collaborative tasks like Lego building, robots that accounted for their partner's point of view achieved significantly higher joint performance, even if the human's subjective rating of the robot did not immediately change.

However, a persistent challenge remains in the realm of kinematics. While humans easily distinguish a social gesture from a functional reach based on the quality of movement, standard machine learning techniques still struggle to perceive this "social quality," focusing instead on raw trajectories.

Future Directions

The authors conclude that while progress has been made, most current models remain too data-driven or static. Future work must focus on deepening social cognition by modeling the processes of human interaction rather than just labeling emotions. This requires a tighter integration of developmental psychology and neuroscience into robotics, ensuring that robots do not just see an emotion, but understand the history and context that produced it.

Summary updated

Abstract

EDITORIAL article Front. Integr. Neurosci., 09 September 2022 https://doi.org/10.3389/fnint.2022.1024267
BibTeX
@inproceedings{sciutti2022editorial,
  author = {Alessandra Sciutti and Pablo Barros and Ginevra Castellano and Yukie Nagai and Pablo V. A. Barros},
  title = {{Editorial: Affective shared perception}},
  booktitle = {Frontiers in Integrative Neuroscience},
  year = {2022},
  doi = {10.3389/fnint.2022.1024267},
}

Loading related content...

Figure