A neural model of how the brain computes heading from optic flow in realistic scenes

N. Andrew Browning, Stephen Grossberg, Ennio MingollaCognitive 2009 · DOI

TL;DR

This paper introduces ViSTARS, a neural model that replicates how primates and humans compute heading from optic flow in complex natural scenes. The model not only achieves human-like accuracy in estimating direction of travel but also provides a biologically plausible explanation for the underlying neural mechanisms.

What Problem Does This Paper Address?

Navigating through our environment is a fundamental cognitive ability. Animals, including humans, rely heavily on visual information, specifically optic flow, to determine their direction of travel (heading), avoid obstacles, and reach goals. While many models exist that can either predict navigational behavior or demonstrate competence in simulated environments, few successfully bridge the gap by explaining both the behavioral data and the underlying neural substrates. This paper aims to address this by developing a neural model, ViSTARS, that can process realistic visual scenes and quantitatively match human heading estimation data, while being grounded in the known neurophysiology of the primate visual system.

Methods

The ViSTARS model is a multi-layered neural network designed to mimic the primate magnocellular pathway, from retinal input up to cortical areas V1, MT⁺, and MSTd. Key aspects of its methodology include:

  • Hierarchical Processing: The model processes visual input through multiple scales, starting with ON-center OFF-surround processing in Level 1 to enhance spatial discontinuities.
  • Directional Selectivity: Levels 2 and 3 introduce transient and directionally selective cells, respectively, to detect motion in specific directions and speeds, addressing the aperture problem.
  • Feature Enhancement and Competition: Level 4 uses a short-range filter and cross-directional normalization to enhance unambiguous motion signals and suppress ambiguous ones, improving the robustness of local motion estimates.
  • Long-Range Integration and Feedback: Level 5 (MT⁺) integrates motion estimates across space and time using a directional long-range filter, and incorporates feedback from Level 6 (MSTd) to refine global motion estimates.
  • Adaptive Flow Filters (Templates): Level 6 (MSTd) uses adaptive flow filters (templates) that represent idealized motion patterns for specific headings. The model computes heading as a distributed population code across these filters.
  • Shunting and Choice Networks: The model extensively utilizes shunting (membrane) equations for normalization and competition, and choice (winner-take-all) networks for selecting the most likely heading.

Datasets

The ViSTARS model was evaluated on a variety of datasets:

  • Random Dot Stimuli: Including ground plane, 3D dot cloud, and frontal plane stimuli, with and without simulated rotations. These were created to mimic existing psychophysical experiments.
  • Yosemite Sequence: A natural image sequence used to test the model's robustness to noise in realistic scenes.
  • OpenGL Stimuli: Computer-generated terrain sequences designed to be challenging due to their homogeneous visual appearance.
  • Driving Video Clips: Real-world video footage from a car, categorized by maneuver type (left turn, bend, lane change, straight).

Key Insights and Ideas

  • Heading as a Distributed Representation: The model demonstrates that accurate heading can be estimated from a distributed population code in MSTd cells, rather than relying on a single "heading cell."
  • Robustness to Noise: The combination of spatiotemporal averaging and competitive interactions within the model makes heading estimation remarkably robust to high levels of noise in the input stream.
  • Importance of Competition: The study highlights that competitive dynamics within MT⁺ are crucial for stable and accurate heading estimation, especially in challenging visual environments.
  • Integration of Visual Streams: ViSTARS integrates processing from lower visual areas (V1, MT⁺) to higher areas (MSTd) in a way that mirrors primate visual processing, linking low-level motion cues to high-level navigation decisions.
  • Computational Efficiency: The model achieves accurate heading with a relatively small number of templates, suggesting that a simplified representation of space can be sufficient for key human data.

Key Results

  • High Accuracy: The model achieved mean heading errors of 1.2-1.4 degrees on random dot and OpenGL stimuli, with a maximum error of 3.83 degrees.
  • Robustness to Rotation: The model's heading error increased with simulated rotation rates, mirroring human behavior. Rotations of 1 degree per second produced errors comparable to those observed in humans.
  • Noise Tolerance: The model demonstrated good performance on noisy Yosemite sequences, with opponent, distributed-opponent, and orthogonal-opponent competition handling SNRs greater than 1.5.
  • Real-World Performance: On driving video clips, the model achieved a correct classification rate of 96% for a 3-class maneuver grouping and 76% for a more detailed 7-class grouping, with mean heading estimates within 3 degrees of the class center.

Limitations and Future Work

  • Parameter Tuning: While the model's parameters were set to match neurophysiological data, some parameters (like F₁) were tuned to optimize performance on specific stimuli, suggesting a need for more generalized parameterization.
  • Aperture Problem Handling: While the model mitigates the aperture problem, it doesn't explicitly solve it in the same way some other models do. The authors suggest that detailed computation of occlusion and transparency might not be necessary for heading calculation.
  • Log-Polar Mapping: The model does not incorporate log-polar mapping, which is computationally expensive but might be necessary to fully match human steering data in certain scenarios. Future work could explore incorporating space-variant filters.
  • Task-Selective Attention: The model does not account for higher-level cognitive processes like task-selective focusing of spatial attention, which might influence heading estimation in humans.
  • Planar/Spiral Motion Cells: The current model primarily uses radial heading templates. Incorporating planar or spiral motion cells in MSTd could potentially explain more complex human data.
  • Integration with STARS: The ViSTARS model is designed to feed into the STARS model for obstacle avoidance and goal approach, suggesting future work on integrating these systems for complete navigation.

Summary updated

BibTeX
@inproceedings{browning2009neural,
  author = {N. Andrew Browning and Stephen Grossberg and Ennio Mingolla},
  title = {{A neural model of how the brain computes heading from optic flow in realistic scenes}},
  booktitle = {Cognitive},
  year = {2009},
  doi = {10.1016/j.cogpsych.2009.07.002},
}
Figure