Hierarchical processing of complex motion along the primate dorsal visual pathway

Patrick Mineault, Farhan A. Khawaja, Daniel A. Butts, Christopher C. PackProceedings of the National Academy of Sciences 2012 · DOI

TL;DR

Researchers have developed a new computational model that better explains how neurons in the primate visual cortex (area MST) process complex motion. This model reveals that a crucial, non-linear "multiplicative" interaction between inputs from earlier visual areas is key to accurately estimating 3D object velocity, a vital function for navigation and interaction with the environment.

What Problem Does This Paper Address?

The primate visual system processes information hierarchically, with later areas building on the computations of earlier ones. Area MST, part of the dorsal visual pathway, is known to be highly selective for complex motion patterns like expansion, rotation, and deformation, which are crucial for tasks like navigation and understanding object movement. However, the exact computational mechanisms by which MST neurons achieve this complex selectivity have remained elusive. Previous attempts to model MST responses using simple linear combinations of inputs from the preceding area (MT) have fallen short, suggesting that more sophisticated processing is at play. This paper aims to bridge this gap by developing and testing a more refined hierarchical model that accounts for the complex selectivity observed in MST neurons.

Methods

The authors employed a multi-pronged approach:

  • Novel Stimulus Design: They created a unique, naturalistic optic flow stimulus that comprehensively sampled the space of complex motion patterns, allowing for a more thorough probing of MST neuron selectivity than previous, simpler stimuli.
  • Computational Modeling: They developed a hierarchical model where inputs from the MT area were processed by simulated "subunits" and then integrated to produce an MST neuron's output.
  • Model Comparison: They systematically compared different model architectures, including a basic linear hierarchical model and a more complex model incorporating a non-linear integration step.
  • Non-linear Integration: The core innovation was the introduction of a non-linear integration mechanism, specifically a compressive (power-law) nonlinearity, applied to the outputs of the MT subunits before they were summed by the MST neuron. This nonlinearity was found to approximate a multiplicative interaction.
  • Population Decoding Simulations: To assess the functional significance of their findings, they used their model MST neurons to estimate 3D object velocity and compared the performance of models with and without non-linear integration.

Datasets

  • Experimental Data: The study utilized electrophysiological recordings from 61 neurons in the medial superior temporal (MST) area of two awake, fixating macaque monkeys.
  • Stimulus Sets: Two types of stimuli were used:
  • Tuning Curve Stimuli: A set of 24 stimuli representing translation, spirals, and deformation, presented at nine spatial positions.
  • Continuous Optic Flow Stimulus: A more complex, naturalistic stimulus consisting of continuously evolving, random combinations of translation, spirals, and deformation, used for model fitting.

Key Insights and Ideas

  • Non-linear Integration is Crucial: The most significant finding is that a simple linear combination of inputs from MT is insufficient to explain MST neuron selectivity. A non-linear integration step, specifically one that approximates multiplicative interactions, is essential.
  • Compressive Non-linearity: The required non-linearity is predominantly compressive, meaning that as input strength increases, the output's rate of increase slows down. This is functionally similar to how logarithms work.
  • Functional Role in 3D Velocity Estimation: This non-linear integration mechanism is not just an arbitrary computational step; it directly improves the ability to estimate 3D object velocity, particularly the depth component (z-velocity), which is strongly related to optic flow patterns like expansion.
  • Hierarchical Structure is Validated: The study provides strong support for a hierarchical processing scheme, where MST neurons integrate information from MT, but refines our understanding of how this integration occurs.
  • Diversity in Subunit Structure: While the non-linear integration mechanism appears conserved, the specific arrangement and tuning of the MT subunits feeding into an MST neuron can be quite diverse, suggesting that MST neurons are not simply detectors of specific optic flow patterns but rather flexible integrators.

Key Results

  • A hierarchical model with a single, shared compressive non-linearity (approximating multiplicative interaction) significantly improved the prediction of MST neuron responses compared to a purely linear hierarchical model.
  • The non-linear integration model improved prediction accuracy (measured by R²) for 75% of the tested MST cells, with a median improvement from 0.31 to 0.50.
  • In population decoding simulations, the non-linear integration model reduced the error in estimating 3D object velocity by 60% compared to the linear model, especially for the depth component.
  • The analysis of receptive field substructures revealed that while the non-linear integration mechanism was consistent, the specific arrangement of MT subunits varied considerably between MST neurons.

Limitations and Future Work

  • Subunit Correspondence: The model's "subunits" are computational constructs and may not perfectly correspond to individual anatomical inputs from MT neurons.
  • Surround Suppression: While the study explored surround suppression as a potential mechanism for non-linearity, it did not consistently outperform the simpler non-linear integration model. However, the authors suggest that surrounds might play a subtle role.
  • Naturalistic Scenes: The current study focused on optic flow stimuli. Future work could explore how these mechanisms generalize to more complex naturalistic scenes with multiple objects and observer motion.
  • Temporal Dynamics: The current model primarily focuses on static nonlinearities. Investigating the temporal dynamics of these non-linear interactions could provide further insights.
  • Physiological Mechanisms: While the compressive non-linearity is functionally important, the precise physiological mechanisms (e.g., synaptic depression, self-normalization) remain to be definitively identified.

Summary updated

Abstract

Neurons in the medial superior temporal (MST) area of the primate visual cortex respond selectively to complex motion patterns defined by expansion, rotation, and deformation. Consequently they are often hypothesized to be involved in important behavioral functions, such as encoding the velocities of moving objects and surfaces relative to the observer. However, the computations underlying such selectivity are unknown. In this work we have developed a unique, naturalistic motion stimulus and used it to probe the complex selectivity of MST neurons. The resulting data were then used to estimate the properties of the feed-forward inputs to each neuron. This analysis yielded models that successfully accounted for much of the observed stimulus selectivity, provided that the inputs were combined via a nonlinear integration mechanism that approximates a multiplicative interaction among MST inputs. In simulations we found that this type of integration has the functional role of improving estimates of the 3D velocity of moving objects. As this computation is of general utility for detecting complex stimulus features, we suggest that it may represent a fundamental aspect of hierarchical sensory processing.
BibTeX
@inproceedings{mineault2012hierarchical,
  author = {Patrick Mineault and Farhan A. Khawaja and Daniel A. Butts and Christopher C. Pack},
  title = {{Hierarchical processing of complex motion along the primate dorsal visual pathway}},
  booktitle = {Proceedings of the National Academy of Sciences},
  year = {2012},
  doi = {10.1073/pnas.1115685109},
}
Figure