Causal inference during closed-loop navigation: parsing of self- and object-motion

Jean‐Paul Noel, Johannes Bill, Haoran Ding, John J. Vastola, Gregory C. DeAngelis, Dora E. Angelaki, Jan DrugowitschPhilosophical 2023

TL;DR

This paper investigates how our brains distinguish between self-motion and object-motion when navigating in real-time. By developing a computational model and conducting virtual reality experiments, the researchers found that humans often misattribute object motion to their own movement, especially during passive self-motion or when targets are off-center. Eye-tracking data further revealed a temporal cascade in this causal inference process, with initial movements being more accurate than later ones.

What Problem Does This Paper Address?

Understanding how we perceive and interact with our environment relies heavily on our ability to infer the causes of sensory signals. A key aspect of this is Causal Inference (CI), where we figure out what's responsible for what we see, hear, or feel. While CI has been extensively studied in simple, static tasks (like deciding which of two options is correct), it's less understood in dynamic, real-world scenarios where we're constantly moving and interacting with our surroundings – what the authors call "closed-loop action-perception loops." This paper specifically tackles the challenge of disentangling retinal motion caused by our own movement (self-motion) from motion caused by external objects, a crucial computation for effective navigation.

Methods

The researchers employed a two-pronged approach:

  • Normative Modeling: They first developed a computational model that predicts how an ideal observer should behave when trying to intercept a moving target. This model uses Bayesian inference to estimate the target's motion and then plans a trajectory to intercept it. This provides a theoretical benchmark for human performance.
  • Virtual Reality Experiments: Human participants navigated in a virtual environment using a joystick. The environment provided optic flow cues (visual patterns that indicate motion) when participants moved. Targets were presented briefly, and participants had to intercept them. Crucially, the experiment manipulated whether participants were moving ("self-motion") or stationary ("no self-motion") during target presentation, and whether the target itself was moving.
  • Eye-Tracking: To capture the temporal dynamics of CI, eye movements (saccades and smooth pursuit) were recorded during the task. This allowed the researchers to see how participants initially oriented themselves and then tracked the target.

Datasets

Not discussed in the paper. The study used a custom-built virtual reality environment and collected data from human participants.

Key Insights and Ideas

  • Self-Motion is a "Noisy Neighbor": When we're moving, the visual signals (optic flow) generated by our own motion can "mask" or interfere with the signals from moving objects. This makes it harder to accurately perceive the object's true motion.
  • Misattribution is Common: The study found that humans frequently misattribute the motion of an object to their own self-motion. This means they might perceive a stationary object as moving, or underestimate the speed of a moving object, because their brain is trying to account for their own movement.
  • Passive Motion is Trickier: When self-motion is passive (e.g., being on a moving walkway where you don't actively control your movement), the brain has less information (like efference copies) to distinguish it from object motion. This leads to even greater misattribution.
  • Eccentric Targets are More Confusing: Targets presented in the periphery of our vision are more likely to be misattributed, likely because the optic flow patterns associated with self-motion are more complex and varied in the periphery.
  • A Temporal Cascade of CI: The analysis of eye movements suggests that CI isn't an instantaneous process. Initial eye movements (saccades) are relatively accurate, indicating the brain can quickly parse some information. However, later smooth pursuit eye movements are less modulated by target motion when self-motion is present, suggesting a more protracted and less precise inference process over time.

Key Results

  • Stationary Reports: Participants were more likely to report moving targets as stationary when they themselves were moving. This effect was more pronounced with longer target presentation times and higher object velocities.
  • Steering Behavior: Participants' steering trajectories indicated they often aimed for the initial position of a moving target rather than its final position when they were moving, suggesting they perceived the target as moving slower or not at all.
  • Passive vs. Active Self-Motion: Misattribution of object motion was significantly greater during passive self-motion compared to active self-motion.
  • Central vs. Eccentric Targets: Misattribution was also exacerbated when targets were presented eccentrically compared to centrally.
  • Eye Movements: While initial saccades to the target were largely accurate regardless of self-motion, smooth pursuit eye movements were modulated by target velocity in the absence of self-motion but not when self-motion was present.

Limitations and Future Work

  • Focus on Specific Events: Much of the analysis relied on specific points in time (e.g., trajectory endpoints, explicit reports), which is a limitation in studying continuous dynamic processes.
  • Modeling Challenges: Jointly modeling CI, path integration, and control dynamics is complex.
  • Single Inference Event: In the current paradigm, causal inference about object motion occurred primarily during the brief observation period.
  • Future Directions: The authors suggest leveraging full trajectories to model evolving beliefs and incorporating optic flow vectors directly into models. They also propose extending this work to the neural level, investigating how single neurons contribute to this process.

Summary updated

Abstract

A key computation in building adaptive internal models of the external world is to ascribe sensory signals to their likely cause(s), a process of Bayesian Causal Inference (CI). CI is well studied within the framework of two-alternative forced-choice tasks, but less well understood within the cadre of naturalistic action-perception loops. Here, we examine the process of disambiguating retinal motion caused by self- and/or object-motion during closed-loop navigation. First, we derive a normative account specifying how observers ought to intercept hidden and moving targets given their belief over (i) whether retinal motion was caused by the target moving, and (ii) if so, with what velocity. Next, in line with the modeling results, we show that humans report targets as stationary and steer toward their initial rather than final position more often when they are themselves moving, suggesting a misattribution of object-motion to the self. Further, we predict that observers should misattribute retinal motion more often: (i) during passive rather than active self-motion (given the lack of an efference copy informing self-motion estimates in the former), and (ii) when targets are presented eccentrically rather than centrally (given that lateral self-motion flow vectors are larger at eccentric locations during forward self-motion). Results confirm both of these predictions. Lastly, analysis of eye-movements show that, while initial saccades toward targets are largely accurate regardless of the self-motion condition, subsequent gaze pursuit was modulated by target velocity during object-only motion, but not during concurrent object- and self-motion. These results demonstrate CI within action-perception loops, and suggest a protracted temporal unfolding of the computations characterizing CI.
BibTeX
@inproceedings{noel2023causal,
  author = {Jean‐Paul Noel and Johannes Bill and Haoran Ding and John J. Vastola and Gregory C. DeAngelis and Dora E. Angelaki and Jan Drugowitsch},
  title = {{Causal inference during closed-loop navigation: parsing of self- and object-motion}},
  booktitle = {Philosophical},
  year = {2023},
}
Figure