TL;DR
This paper presents a novel deep active inference model that successfully replicates the human rubber-hand illusion (RHI) in a virtual environment. By integrating visual and proprioceptive information, the model not only reproduces the perceptual drift associated with the RHI but also generates realistic involuntary forces, offering a computational framework for understanding body perception and action under sensorimotor conflict.
What Problem Does This Paper Address?
Seamless interaction with our environment relies on the brain's ability to integrate noisy sensory information from multiple modalities into a coherent world model. A critical challenge in this process is handling sensorimotor conflicts – situations where different senses provide contradictory information about the body's state or its interaction with the world. The rubber-hand illusion (RHI) is a classic experimental paradigm that highlights this challenge. In the RHI, participants experience a sense of ownership over a rubber hand that is visually presented in a location different from their own real hand, leading to a perceived drift of their real hand towards the rubber hand. Recent research has also shown that humans exhibit involuntary forces in response to the RHI, suggesting a deep interdependence between perception and action. This paper aims to bridge this gap by developing a computational model that can account for both the perceptual and motor aspects of the RHI, offering insights into how the brain resolves such conflicts.
Methods
The authors propose a deep active inference agent operating in a 3D virtual reality (VR) environment. The core of their approach is an active inference framework, which formalizes perception and action as an inference problem. The agent infers its unobserved body state by minimizing a variational free-energy bound. This inference process utilizes two sensory modalities: visual input (high-dimensional pixel matrices from the VR environment) and proprioceptive information (joint angles of the virtual arm).
Key methodological aspects include:
- Deep Generative Models: To handle high-dimensional visual inputs, the model employs deep neural networks to approximate the visual generative process and its derivatives. They compare two architectures: a convolutional decoder and a variational autoencoder (VAE).
- Active Inference Equations: The model's dynamics are described by a system of differential equations derived from the free-energy principle, governing the evolution of brain variables (representing body state), prediction errors, and actions.
- Visuo-Tactile Synchrony Modeling: To simulate the RHI, the model incorporates a mechanism to weigh the causal influence of visual and tactile stimulation. This is achieved using a Bayesian causal model that assigns a probability to two explanations: either the observed (virtual) hand produced both events, or the visual event was from the virtual hand and the tactile event from the real hand. This leads to an update rule for a parameter ($\gamma$) that modulates the influence of visual information.
Datasets
The paper does not explicitly mention training on pre-existing datasets. Instead, the model is trained and evaluated within a custom-built virtual environment using the Unity engine. The visual input is generated by a camera within this environment, producing 256x256 grayscale images. The proprioceptive input is derived from the joint angles of the agent's virtual arm.
Key Insights and Ideas
- Unified Perception-Action Framework: The paper demonstrates that active inference, a framework that unifies perception and action, can effectively model complex sensorimotor phenomena like the RHI. This suggests that the brain might resolve perceptual conflicts by simultaneously inferring body state and generating appropriate actions.
- Deep Learning for High-Dimensional Sensory Input: The successful integration of deep neural networks (CNNs and VAEs) to process raw visual data is crucial. This allows the model to learn rich representations of the visual environment, a significant step towards more biologically plausible and capable artificial agents.
- Causal Inference for Sensorimotor Conflict: The use of Bayesian causal inference to weigh conflicting sensory inputs provides a principled way to handle situations like the RHI, where the perceived cause of sensory events is ambiguous.
- Emergence of Involuntary Forces: The model's ability to generate realistic involuntary forces in response to the RHI is a key finding. This suggests that these forces are not merely a byproduct but an integral part of the brain's predictive processing and action generation mechanisms when dealing with uncertainty.
Key Results
- Perceptual Drift Replication: The model successfully replicates the perceptual drift of the end-effector location observed in human RHI experiments (Fig. 3a, b, c).
- Action Pattern Replication: The model generates horizontal end-effector accelerations that align with human behavioral data (Fig. 3d, e, f).
- Force Generation: The model produces forces that are comparable to those observed in human participants during synchronous stimulation (Fig. 3g). Notably, the VAE-based visual model produced smoother and more bounded visual Jacobian values, suggesting a more robust internal representation.
- Asynchronous Stimulation Effects: Asynchronous visuo-tactile stimulation resulted in attenuated forces, mirroring human observations.
- Comparison of Visual Models: The VAE-based visual model generally performed better, producing more consistent and less "non-meaningful" forces compared to the convolutional decoder, likely due to its probabilistic latent space.
Limitations and Future Work
- Simplified Body Dynamics: The current model simplifies the body state to joint angles and assumes no body movement during the passive RHI experiment. Future work could explore more complex body dynamics and active exploration.
- Limited Sensory Modalities: While the model incorporates vision and proprioception, future research could extend it to include other modalities like touch and audition.
- Investigating Sensorimotor Self: The authors suggest that this model can be further employed to investigate the construction of the "sensorimotor self," which is a promising avenue for future research.
- Generalizability: While the model successfully replicates the RHI, further validation on other sensorimotor conflict scenarios would be beneficial.
Summary updated