Sensory Evidence Accumulation Using Optic Flow in a Naturalistic Navigation Task

Panos Alefantis, Kaushik J. Lakshminarasimhan, Eric Avila, Jean‐Paul Noel, Xaq Pitkow, Dora E. Angelaki

TL;DR

Researchers have long debated whether we navigate by "dead reckoning" (relying on an internal model of our own actions) or by continuously integrating sensory feedback. By using a closed-loop virtual reality task, this study demonstrates that both humans and macaques rely heavily on optic flow—the visual motion of the environment—to accumulate evidence and update their position in real-time.

What Problem Does This Paper Address?

In the real world, perception and action are inseparable: your movements change what you see, and what you see informs your next movement. This is a "closed-loop" system. However, most neuroscience research on "evidence accumulation" (how the brain gathers information to make a decision) uses "open-loop" tasks where subjects passively watch stimuli (like moving dots) and make a single binary choice.

The challenge in studying closed-loop navigation is ambiguity. If a subject reaches a target using a joystick, are they actually "reading" the visual flow of the ground beneath them, or are they simply counting how long they’ve held the joystick forward based on a learned internal model? This paper aims to prove that the brain actively integrates continuous sensory evidence during naturalistic movement.

Methods

The authors developed the "Firefly Task":

  1. The Goal: Subjects (3 macaques and ~33 humans) used a joystick to navigate to a target (a "firefly") that flashed briefly on a ground plane in VR and then disappeared.
  2. The Stimulus: The environment was devoid of landmarks. The only cues were "optic flow" elements—small triangles on the ground that appeared and disappeared quickly to prevent them from being used as fixed landmarks.
  3. The Manipulations: To decouple sensory input from the subjects' expectations, the researchers used three "interventions":
  4. Perturbations: Randomly "pushing" the subject off-course with unexpected visual motion.
  5. Joystick Gain: Changing the "sensitivity" of the joystick (e.g., making the joystick move the subject 1.5x or 2x faster than usual).
  6. Density/Timing: Varying the number of ground elements or the duration they were visible to change the "quality" of the sensory evidence.

Datasets

The study generated original behavioral datasets:

  • Non-human primates: Data from 3 rhesus macaques.
  • Humans: Data from four distinct groups (totaling 33 subjects) participating in different task variants (perturbation, gain, density, and time-varying density).
  • Code Availability: The authors provided their analysis code on GitHub and the raw datasets on the G-Node infrastructure.

Key Insights and Ideas

  • Sensory Integration is Continuous: By showing that performance improved as the duration of optic flow increased (from 500ms to 1500ms), the authors proved that humans don't just use a "snapshot" to calibrate their speed; they accumulate evidence throughout the entire journey.
  • The Brain Compensates for "Lies": When the joystick gain was secretly increased, subjects didn't just overshoot the target. They used the optic flow to realize they were moving faster than expected and stopped earlier.
  • Cross-Species Conservation: The fact that both macaques and humans showed similar compensatory behaviors suggests that the neural circuits for optic flow integration are evolutionarily conserved and that macaques are an excellent model for studying human navigation.
  • Internal Models vs. Sensory Evidence: While subjects relied heavily on optic flow, their compensation wasn't always 100% perfect. This suggests a "weighted" strategy where the brain balances sensory evidence against its internal expectations (efference copies) of the motor command.

Key Results

  • Perturbation Compensation Index (PCI): Humans showed near-ideal compensation (~0.89), while macaques were slightly lower but still significant (~0.61). Both were far better than a "no-compensation" model.
  • Gain Compensation Index (GCI): In the gain manipulation task, monkeys adjusted their responses almost immediately (within the first 20 trials of a new block), with GCIs ranging from 0.67 to 0.77.
  • Accuracy vs. Information: Removing optic flow entirely or reducing its density significantly increased the standard deviation of stopping positions (e.g., in humans, angular error jumped from ~14° with optic flow to ~40° without it).
  • ROC Analysis: Using Receiver Operating Characteristic (ROC) curves, the authors showed that the "Area Under the Curve" (a measure of accuracy) was significantly higher in trials with sensory feedback compared to simulated "uncompensated" trials ($p < 10^{-6}$).

Limitations and Future Work

  • The "Where" is Missing: This paper focuses entirely on behavior. While it establishes that evidence accumulation happens in closed-loop navigation, it doesn't show where in the brain this occurs (though it points toward the posterior parietal cortex).
  • Sub-optimal Macaque Performance: Macaques were less "ideal" than humans in compensating for perturbations. The authors suggest this might be due to humans having a more sophisticated ability to decouple self-motion from external environmental changes.
  • Future Direction: The authors suggest that future studies should manipulate "motor noise" (making the joystick itself shaky) to see if subjects rely even more on sensory evidence when their own actions become unreliable.

Summary updated

Abstract

Sensory evidence accumulation is considered a hallmark of decision-making in noisy environments. Integration of sensory inputs has been traditionally studied using passive stimuli, segregating perception from action. Lessons learned from this approach, however, may not generalize to ethological behaviors like navigation, where there is an active interplay between perception and action. We designed a sensory-based sequential decision task in virtual reality in which humans and monkeys navigated to a memorized location by integrating optic flow generated by their own joystick movements. A major challenge in such closed-loop tasks is that subjects' actions will determine future sensory input, causing ambiguity about whether they rely on sensory input rather than expectations based solely on a learned model of the dynamics. To test whether subjects integrated optic flow over time, we used three independent experimental manipulations, unpredictable optic flow perturbations, which pushed subjects off their trajectory; gain manipulation of the joystick controller, which changed the consequences of actions; and manipulation of the optic flow density, which changed the information borne by sensory evidence. Our results suggest that both macaques (male) and humans (female/male) relied heavily on optic flow, thereby demonstrating a critical role for sensory evidence accumulation during naturalistic action-perception closed-loop tasks.<b>SIGNIFICANCE STATEMENT</b> The temporal integration of evidence is a fundamental component of mammalian intelligence. Yet, it has traditionally been studied using experimental paradigms that fail to capture the closed-loop interaction between actions and sensations inherent in real-world continuous behaviors. These conventional paradigms use binary decision tasks and passive stimuli with statistics that remain stationary over time. Instead, we developed a naturalistic visuomotor visual navigation paradigm that mimics the causal structure of real-world sensorimotor interactions and probed the extent to which participants integrate sensory evidence by adding task manipulations that reveal complementary aspects of the computation.
BibTeX
@article{alefantis2022sensory,
  author = {Panos Alefantis and Kaushik J. Lakshminarasimhan and Eric Avila and Jean‐Paul Noel and Xaq Pitkow and Dora E. Angelaki},
  title = {{Sensory Evidence Accumulation Using Optic Flow in a Naturalistic Navigation Task}},
  journal = {Journal of Neuroscience},
  year = {2022},
  doi = {10.1523/jneurosci.2203-21.2022},
}
Figure