Introduction: The Correspondence Problem
In the field of neuroscience, the discovery of "mirror neurons"—cells that fire both when an individual performs an action and when they observe another performing the same action—sparked a significant debate. While these neurons suggest a deep link between perception and action, they do not explain how the brain bridges the gap between a visual signal from an external vantage point and its own internal motor representations. This is known as the correspondence problem.
Traditional artificial intelligence approaches often solve this through supervised learning, requiring massive datasets of actions viewed from every possible angle. However, human cognition appears to be more embodied. We likely use our own sensory-motor experiences as a template to interpret others. This paper, published in Frontiers in Computational Neuroscience, proposes a generative neural model that learns from its own movement to solve the correspondence problem through active inference and mental rotation.
The Proposed Architecture
The authors developed a hierarchical processing cascade consisting of three distinct stages. This architecture is designed to handle both visual input (observing others) and proprioceptive input (the internal sense of one's own body position).
- Stage I (Feature Processing): This stage converts raw motion data into relative positions and velocities. By focusing on relative rather than absolute coordinates, the model achieves invariance to the subject's location in space and body size. A critical component here is the Perspective-Taking Module, which allows the model to perform 3D rotations on incoming visual data.
- Stage II (Population Coding): Directional signals from Stage I are mapped onto populations of direction-selective neurons. This mimics the way the biological brain represents sensory data, using groups of neurons to encode specific features of movement.
- Stage III (Spatio-temporal Learning): This stage serves as the model's cognitive core. It employs an unsupervised clustering algorithm to segment continuous motion into discrete "patterns." Furthermore, it learns the transition probabilities between these patterns, creating a generative model capable of predicting future movements.
Active Inference and Perspective-Taking
The model’s ability to understand actions from novel viewpoints relies on Active Inference. When the model observes a movement from an unfamiliar angle, its internal predictions (based on its own learned movements) fail to match the visual observation. This creates a "prediction error."
This error is propagated back down the hierarchy to the Perspective-Taking Module (shown in Figure 3A). The module then iteratively adjusts the internal 3D rotation of the visual input until the prediction error is minimized. In effect, the model "mentally rotates" the observed person until their movement matches a known internal template. This mechanism allows the model to generalize its self-knowledge to others without needing exhaustive training on different viewpoints.
Results and Neuro-Cognitive Insights
The model was evaluated using high-fidelity motion tracking data (120 Hz) from three human subjects performing actions like walking, running, and basketball dribbling. The results demonstrated several key strengths:
- Robust Convergence: In 97% of test trials, the model successfully identified the correct perspective from a random starting orientation. This convergence typically occurred within 230 time steps, or roughly two seconds of observed motion.
- Action Discrimination: The model successfully distinguished between 12 different view-dependent movements, encoding them into distinct neural populations.
- Bistable Perception: The model replicated a classic human psychological phenomenon. When presented with ambiguous 2D "point-light walkers," the model’s perception would spontaneously flip between seeing the walker as moving toward or away from it, mirroring human bistable perception.
- Cross-Modal Inference: Because the model learns the correlation between sight and feel, it can infer the internal joint angles (proprioception) of an observed person based solely on their external visual movement.
Conclusion and Future Directions
This research provides a computationally plausible explanation for how embodied agents can understand the actions of others. By treating perspective as a variable to be optimized through error minimization, the model avoids the need for massive, labeled datasets.
However, limitations remain. The current model relies on a pre-defined body structure, requiring researchers to select which markers constitute specific features. Future work aims to develop models that can autonomously learn the hierarchical structure of a body. Additionally, while the model excels at handling 3D rotations, integrating scale invariance (handling different distances) into the active inference loop remains a goal for future iterations.
Summary updated