The development of active binocular vision under normal and alternate rearing conditions

Lukas Klimmasch, Johann Schneider, Alexander Lelais, Maria Fronius, Bertram E. Shi, Jochen Triesch

TL;DR

This paper presents a computational model that simulates the development of active binocular vision in agents that learn to control their eye movements (vergence) while simultaneously developing visual representations. The model, based on Active Efficient Coding, successfully reproduces experimental findings from altered rearing conditions and makes novel predictions about how vergence control influences neural representations.

What Problem Does This Paper Address?

Biological vision systems learn to perceive the world with remarkable autonomy, a stark contrast to current machine learning approaches that often rely on massive labeled datasets. The development of binocular vision, in particular, is a complex process involving both the learning of visual representations (like disparity-tuned neurons) and the refinement of motor control (vergence eye movements). While previous models have explored efficient coding for visual input, they often overlooked the crucial role of active behavior, specifically vergence, in shaping visual development. This paper aims to bridge this gap by presenting a unified model that integrates active perception and motor learning to explain how binocular vision develops and how it is affected by altered sensory input during critical developmental periods.

Methods

The core of the model is an Active Efficient Coding (AEC) framework. This framework combines two main components:

  1. Sparse Coding: A neural network learns to encode visual input from both eyes by adapting its receptive fields (RFs) to efficiently represent the statistical regularities of the input. This component mimics the development of disparity-tuned neurons in the visual cortex.
  2. Reinforcement Learning (RL): An actor-critic architecture learns to control the agent's vergence eye movements. The RL agent receives input from the sparse coding layer and is rewarded for actions that lead to more efficient encoding of the visual scene (i.e., better image alignment and reduced reconstruction error).

The model uses a two-scale approach for patch extraction (coarse and fine) to capture both large and small disparities, inspired by the varying receptive field sizes in the human retina. To simulate altered rearing conditions, input images were convolved with Gaussian filters to mimic specific sensory deprivations (e.g., orientation filtering, monocular deprivation, strabismus).

Datasets

The primary dataset used for training the model was the McGill Calibrated Colour Image Database, specifically the "man-made section." For comparison and to assess the impact of image statistics, a random sample of images from the entire McGill database was also used. The model was evaluated on random-dot stereograms (RDSs), which were generated by the authors.

Key Insights and Ideas

  • Behavior is Key to Visual Development: The model demonstrates that active control of eye movements (vergence) is not just a consequence of visual processing but actively shapes the development of neural representations, particularly disparity tuning.
  • Unified Learning Framework: The AEC framework, by jointly optimizing sensory coding and motor control, provides a powerful mechanism for understanding how biological systems can learn complex perceptual abilities autonomously.
  • Predicting Effects of Altered Rearing: The model successfully reproduces known neurophysiological findings from various altered rearing conditions, validating its ability to capture critical aspects of visual plasticity.
  • Vergence Influences Orientation Tuning: A surprising prediction is that impaired vergence control can systematically alter the statistics of orientation tuning in visual cortical neurons, suggesting a deeper interplay between motor and sensory systems than previously appreciated.
  • Critical Period Importance: The model shows that visual deficits incurred during simulated critical periods persist even after visual input is corrected, mirroring findings in biological systems.

Key Results

  • Normal Rearing: Under normal conditions, the model autonomously learns accurate vergence control, achieving a corrected stereo acuity of 15.9 ± 21.44 arc sec, comparable to human performance. It also successfully interprets random-dot stereograms without explicit training.
  • Altered Rearing Conditions:
    • Orientation Tuning: The model qualitatively reproduces changes in orientation tuning observed in experiments (e.g., loss of vertical/horizontal tuning when those orientations are deprived).
  • Binocularity: The model shows a strong peak in binocularity indices near zero for normal rearing, decreasing with altered conditions, consistent with biological data.
  • Disparity Tuning: Normal rearing leads to the most disparity-tuned cells, while uncorrelated input (strabismus) drastically reduces this number.
  • Vergence Performance: Models trained under orthogonal, monocular, or strabismic conditions show significantly impaired vergence learning, often failing to improve beyond random performance.
  • Aniseikonia: The model predicts an inverted U-shaped relationship between the degree of aniseikonia and the number of neurons tuned to vertical disparities. It also shows that stereo acuity declines quadratically with increasing aniseikonia, matching human experimental data.
  • Vergence and Orientation: Disabling vergence learning leads to a significant decrease in vertically tuned neurons, with the "zero disparity" model showing the fewest. The model predicts a peak in vertically tuned neurons at a specific level of disparity variance.

Limitations and Future Work

  • Simplified Input: The model uses a single, large, frontoparallel textured plane, which is a simplification of natural visual environments. This may contribute to narrower distributions of preferred disparities and orientations compared to biological data.
  • No Interocular Suppression: The model does not incorporate an interocular suppression mechanism, which is thought to play a role in conditions like amblyopia and may explain some quantitative differences with human data.
  • Future Work: The authors suggest extending the model to study amblyopia more deeply, incorporating efference copies of motor commands for more complex tasks like pursuit eye movements, and exploring other sensory modalities. They also propose testing their predictions regarding the influence of vergence on orientation tuning in future experiments.

Summary updated

Abstract

The development of binocular vision is an active learning process comprising the development of disparity tuned neurons in visual cortex and the establishment of precise vergence control of the eyes. We present a computational model for the learning and self-calibration of active binocular vision based on the Active Efficient Coding framework, an extension of classic efficient coding ideas to active perception. Under normal rearing conditions with naturalistic input, the model develops disparity tuned neurons and precise vergence control, allowing it to correctly interpret random dot stereograms. Under altered rearing conditions modeled after neurophysiological experiments, the model qualitatively reproduces key experimental findings on changes in binocularity and disparity tuning. Furthermore, the model makes testable predictions regarding how altered rearing conditions impede the learning of precise vergence control. Finally, the model predicts a surprising new effect that impaired vergence control affects the statistics of orientation tuning in visual cortical neurons.
BibTeX
@inproceedings{klimmasch2021development,
  author = {Lukas Klimmasch and Johann Schneider and Alexander Lelais and Maria Fronius and Bertram E. Shi and Jochen Triesch},
  title = {{The development of active binocular vision under normal and alternate rearing conditions}},
  booktitle = {eLife},
  year = {2021},
  doi = {10.7554/elife.56212},
}
Figure