Object manifold geometry across the mouse cortical visual hierarchy

Emmanouil Froudarakis, Uri Cohen, Maria Diamantaki, Edgar Y. Walker, Jacob Reimer, Philipp Berens, Haim Sompolinsky, Andreas S. ToliasbioRxiv 2020 · DOI

TL;DR

Mice can learn to recognize objects even when their appearance changes, similar to humans. By recording neural activity across the mouse visual cortex, researchers found that later visual areas are better at representing objects in a way that's easier for the brain to "read out." This improved representation is linked to a more compact and less complex "object manifold" in these areas.

What Problem Does This Paper Address?

We humans are remarkably good at recognizing objects, even when they're seen from different angles, in varying light, or partially obscured. This ability to achieve "invariant object recognition" is thought to arise from how our brains process visual information hierarchically. As signals move up the visual processing stream, the complex patterns of neural activity that represent a single object (the "object manifold") are hypothesized to become simpler and more "linearly decodable." This means that a simple linear classifier, akin to a basic decision rule, could distinguish between different objects. While this idea has been explored in primates, the specific changes in the geometry of these object manifolds along the visual hierarchy and how they enable this invariance remained largely unknown, especially in a model organism amenable to detailed circuit dissection. This paper aims to fill this gap by investigating object manifold geometry in the mouse visual cortex.

Methods

The researchers employed a multi-pronged approach:

  • Behavioral Training: They developed an automated home-cage training system to teach mice a two-alternative forced choice (2AFC) object classification task. This involved mice licking one of two ports based on the object presented on a screen. Crucially, the training involved a variety of object transformations (scale, pose, illumination) and tested for generalization to novel transformations.
  • Large-Scale Neural Recordings: Using two-photon microscopy and mesoscopes, they simultaneously recorded the activity of thousands of neurons across multiple visual areas in awake, head-fixed mice.
  • Decoding Analysis: They used logistic regression classifiers to determine how well object identity could be predicted from the neural activity in different visual areas. This was done with varying numbers of neurons and under different conditions (e.g., with background clutter).
  • Manifold Geometry Analysis: They applied a recently developed theory of "linear separability of manifolds" to quantify the geometry of object representations. This involved measuring the "classification capacity," "manifold radius," and "manifold dimension" for each visual area.

Datasets

The study used synthesized 3D objects rendered in Blender. These objects were presented to mice in video clips with varying parameters (translation, scale, pose, illumination). The dataset consisted of neural responses from thousands of neurons across various visual areas of the mouse brain. No specific benchmark datasets were introduced; rather, the study generated its own stimuli and collected novel neural data.

Key Insights and Ideas

  • Mice are capable of invariant object recognition: The study provides strong evidence that mice can learn to recognize objects even when presented with novel transformations, demonstrating a level of invariance comparable to higher mammals. This is a significant finding for using mice as a model for studying visual object recognition.
  • Later visual areas are "better" for object recognition: Areas like LM, LI, and AL showed significantly higher linearly decodable object information compared to earlier areas like V1. This suggests a hierarchical processing stream where object representations become more refined.
  • Object manifolds become simpler in higher visual areas: The key insight is that the improved decodability in later visual areas is not just about more information, but about how that information is organized. The object manifolds in these areas become smaller (reduced radius) and less complex (reduced dimension).
  • Geometry dictates decodability: The theory of linear separability of manifolds provides a framework to link the geometric properties of object manifolds (radius, dimension) directly to the brain's ability to linearly decode object identity. This offers a mechanistic explanation for why certain areas are better at object recognition.

Key Results

  • Mice trained on a 2AFC task showed consistent performance and generalized to unseen object transformations.
  • Lateral extrastriate visual areas (LM, LI, AL) carried more linearly decodable object information than V1 and other areas.
  • The increase in classification capacity in LM and AL was associated with a decrease in both manifold dimension and radius.
  • Generalization performance across background noise was also better in LM and AL compared to V1.
  • The study found that the dimensionality of neural population responses decreased along the visual hierarchy, consistent with previous findings in primates.

Limitations and Future Work

  • Behavioral Task Necessity: The study notes that while they found variability in LI's performance, it might be due to the experiments being conducted in awake, passive viewing animals without a specific behavioral task that would necessitate LI's engagement. Future experiments using ethologically relevant tasks are suggested.
  • Causal Relationships: To establish a more causal link between visual areas and behavior, the authors suggest combining behavioral performance with causal manipulations of neural activity.
  • Model-Based Approaches: The paper highlights the potential of neural network models and the inception loop methodology to further characterize the specific features driving neuronal populations in different visual areas.
  • Dorsal vs. Ventral Pathways: While the study's findings on AL's performance are somewhat inconsistent with the current assumption of AL being part of a distinct dorsal pathway, the authors acknowledge that strong interactions between dorsal and ventral pathways in rodents might be important for object detection. Further research is needed to fully elucidate the roles of these pathways in mice.

Summary updated

Abstract

Despite variations in appearance, we recognize objects robustly. Neuronal populations responding to objects presented under varying transformations form object manifolds, and hierarchically organized visual areas are hypothesized to untangle pixel intensities into linearly decodable object representations. However, the associated changes in the geometry of the object manifolds along the cortex remain unknown. Using home cage training, we showed that mice are capable of invariant object recognition. We recorded the responses of thousands of neurons to measure the information about object identity across the visual cortex and found that the lateral areas LM, LI, and AL carry more linearly decodable object information compared to other visual areas. We applied the theory of linear separability of manifolds and found that the increase in classification capacity is associated with a decrease in the dimension and radius of the object manifold, identifying the key features in the geometry of the population neural code that are associated with invariant object coding.
BibTeX
@misc{froudarakis2020object,
  author = {Emmanouil Froudarakis and Uri Cohen and Maria Diamantaki and Edgar Y. Walker and Jacob Reimer and Philipp Berens and Haim Sompolinsky and Andreas S. Tolias and Philipp Berens and Haim Sompolinsky and Andreas S. Tolias},
  title = {{Object manifold geometry across the mouse cortical visual hierarchy}},
  year = {2020},
  doi = {10.1101/2020.08.20.258798},
  howpublished = {bioRxiv},
}
Figure