MouseNet: A biologically constrained convolutional neural network model for the mouse visual cortex

Jianghong Shi, Bryan P. Tripp, Eric Shea-Brown, Stefan Mihalas, Michael A. BuicePLOS Computational Biology 2022 · DOI

TL;DR

Researchers have developed MouseNet, a novel convolutional neural network (CNN) architecture that mimics the biological structure of the mouse visual cortex. By incorporating anatomical data, MouseNet achieves a more biologically plausible representation of visual processing, offering a new tool to study how brain structure influences function and task learning.

What Problem Does This Paper Address?

Convolutional neural networks (CNNs) have been incredibly successful in computer vision tasks, often inspired by the primate visual system. However, these models are typically much deeper and more complex than the actual biological visual systems they aim to emulate. This raises questions about how to interpret their success and whether their architectural choices are truly representative of biological computation. The mouse visual system, with its shallower and more parallel structure compared to primates, presents a unique case study. This paper addresses the challenge of building a CNN that reflects the specific architecture of the mouse visual cortex, aiming to bridge the gap between artificial neural networks and biological neural systems.

Methods

The core innovation is the development of MouseNet, a CNN architecture whose structure and parameters are derived from biological data. This includes:

  • Anatomical Constraints: The network's hierarchy and connectivity patterns are based on tract-tracing data and analyses of the mouse brain's visual areas.
  • Neuron Counts and Layer Statistics: Estimates of neuron numbers in different areas and cortical layers, along with connection statistics between these layers, inform the network's design.
  • Mesoscale Connectome: A 100-micrometer resolution interareal connectome dataset was used to shape the spatial profiles and connection strengths of the network's convolutional kernels.
  • Gaussian Masks: To model the spatially varying connection densities observed in the brain, 2D Gaussian masks were applied to the convolutional kernels, introducing biologically inspired sparsity and spatial tuning.
  • Task-Optimized Training: MouseNet was trained on standard image classification tasks (CIFAR10 and ImageNet) to evaluate its computational performance and its ability to learn representations similar to those found in the mouse brain.

Datasets

  • ImageNet Large Scale Visual Recognition Challenge 2012 (ILSVRC2012): Used for training and evaluating the network's image classification performance.
  • CIFAR10: Another dataset used for training and evaluating image classification.
  • Allen Brain Observatory Visual Coding dataset: A large-scale, standardized in vivo survey of physiological activity in the mouse visual cortex, used for comparing MouseNet's representations to biological neural responses.

Key Insights and Ideas

  • Architecture Matters: Building CNNs with biologically constrained architectures, even if smaller than standard models, can lead to representations that are more similar to biological systems.
  • Task Performance vs. Biological Similarity: Achieving higher task performance (e.g., on ImageNet) does not necessarily translate to greater functional similarity with the biological brain. In fact, optimal similarity might be achieved at intermediate training stages.
  • Parallel Pathways: The parallel architecture of the mouse visual cortex, as implemented in MouseNet, appears to lead to more diversified representations within the network after task training, a phenomenon not as pronounced in more monolithic architectures like VGG16.
  • Functional Properties: Task training in MouseNet, guided by its biological constraints, can shift functional properties like lifetime sparseness and orientation selectivity to be more aligned with those observed in the mouse brain.

Key Results

  • Computational Performance: MouseNet achieved over 90% validation accuracy on CIFAR10 and roughly two-thirds the performance of VGG16 on the more challenging ImageNet benchmark, despite being significantly smaller.
  • Representational Similarity: MouseNet showed comparable representational similarity to the mouse brain as VGG16, even though VGG16 had much higher task performance.
  • Training Dynamics: The highest functional similarity between MouseNet and the mouse brain was often observed at intermediate points during training, not necessarily at the point of peak task performance.
  • Functional Properties Alignment: Training MouseNet on image classification brought its distributions of lifetime sparseness and circular selectivity index closer to those observed in the mouse brain data, more so than VGG16.

Limitations and Future Work

  • Task Appropriateness: The authors suggest that ImageNet classification might not be the most appropriate task for fully capturing the computations of the mouse visual cortex, given that optimal biological similarity was not achieved at peak performance. Future work could explore more ecologically relevant tasks for mice, such as motion tracking or predator avoidance.
  • Model Simplifications: The model currently focuses on excitatory neurons and a simplified feedforward pathway. Future work could incorporate inhibitory neurons, recurrence, and more detailed anatomical structures.
  • Generalizability: While MouseNet is a promising framework, further validation across different tasks and species would be beneficial.
  • Future Directions: The authors encourage further research using the MouseNet framework to incorporate new anatomical data, explore the functional significance of specific architectural constraints, and develop more sophisticated models for other species.

Summary updated

BibTeX
@article{shi2022mousenet,
  author = {Jianghong Shi and Bryan P. Tripp and Eric Shea-Brown and Stefan Mihalas and Michael A. Buice},
  title = {{MouseNet: A biologically constrained convolutional neural network model for the mouse visual cortex}},
  journal = {PLOS Computational Biology},
  year = {2022},
  doi = {10.1371/journal.pcbi.1010427},
}
Figure