TL;DR
This paper proposes that the brain's visual system, like deep neural networks, organizes into specialized pathways (ventral for perception, dorsal for action) due to optimizing multiple, distinct cost functions. They demonstrate this by training multi-task neural networks and showing that related tasks lead to shared feature representations, while unrelated tasks lead to specialized ones, mirroring the brain's functional segregation.
What Problem Does This Paper Address?
The long-standing "what" and "where" pathways model of visual processing has evolved into the "vision for perception" (ventral) and "vision for action" (dorsal) pathways. However, the computational principles driving this functional and anatomical segregation remain unclear. This paper bridges this gap by hypothesizing that the brain, much like modern machine learning models, uses multiple cost functions to guide learning. When these cost functions are for related tasks, feature representations are shared across pathways; when they are for unrelated tasks, representations become specialized, leading to distinct pathways. This offers a computational explanation for the observed functional organization of the visual cortex.
Methods
The core of the paper's methodology lies in using multi-task deep neural networks (DNNs) as a computational model for the visual system. They train two versions of the AlexNet architecture:
- RelNN: Trained on two related tasks: subordinate-level object classification and ordinate-level object classification.
- UnrelNN: Trained on two unrelated tasks: subordinate-level object classification and text recognition.
To quantify feature representation sharing, they developed a method to measure the contribution of each unit (feature representation) to each task. This involves marginalizing out a specific parameter (feature representation) and observing the change in classification probability. They then generalize this to measure the contribution of a parameter to an entire task by averaging its contributions to correct class predictions within that task.
Datasets
The paper uses the ImageNet database for training their DNN models. Specifically, they utilize approximately half a million images. For classification, the images are annotated with 15 ordinate classes, further divided into 234 subordinate classes. Additionally, the images are augmented with overlays of 3 letter labels from 15 different classes for the text recognition task.
Key Insights and Ideas
- Cost Functions as the Driving Force: The paper strongly advocates for viewing neural computation through the lens of optimizing cost functions, suggesting this is a fundamental principle for both artificial and biological systems.
- Task Relatedness Dictates Feature Sharing: The degree to which tasks are related directly influences how much their learned feature representations are shared within a network. Related tasks encourage generalizable features, while unrelated tasks promote specialized ones.
- Emergence of Functional Pathways: The segregation of visual processing into ventral and dorsal pathways can be computationally explained by the brain optimizing distinct cost functions for "vision for perception" and "vision for action," which are largely unrelated tasks.
- Hierarchical Representation Sharing: Feature representation sharing is not uniform across network depth. Earlier layers tend to show more sharing (especially for related tasks), while higher layers exhibit more specialization, particularly for unrelated tasks.
- DNNs as a Model for Brain Organization: The paper posits that DNNs, when trained with multi-task learning, can serve as valuable models for understanding the functional organization of the brain, offering a framework to generate testable predictions.
Key Results
- RelNN vs. UnrelNN: The RelNN showed a consistently high degree of feature representation sharing across layers for both tasks. In contrast, the UnrelNN exhibited a decreasing degree of feature sharing towards higher-tier layers, with early layers showing some sharing and later layers becoming more specialized.
- Classification Performance: Table 1 shows the Top-5 error rates for both models. RelNN achieved 14.0% error for subordinate-level recognition and 2.9% for ordinate-level/text recognition. UnrelNN achieved 15.2% for subordinate-level recognition and 4.9% for ordinate-level/text recognition. These results indicate that both models performed reasonably well, allowing for the analysis of their internal representations.
- Visualization of Contributions: Figure 4 visually demonstrates the differential contribution of feature representations to each task. For RelNN, contributions are more evenly distributed and blended (gray/white tones), indicating shared features. For UnrelNN, distinct blue (subordinate-level) and yellow (text) contributions are more apparent, especially in later layers, signifying task-specific representations.
Limitations and Future Work
- Biological Realism of Learning Rules: The paper acknowledges that the precise learning rules of the brain (e.g., backpropagation) are still an open question, and current DNN training methods may not perfectly mirror biological mechanisms.
- Defining Task Relatedness: The paper notes that the relatedness of tasks is not a clearly defined concept and was chosen based on assumptions. Future work could explore more rigorous methods for quantifying task relatedness.
- Catastrophic Forgetting: The authors suggest that multiple pathways, as seen in multi-task DNNs, might offer a solution to catastrophic forgetting, where learning a new task overwrites old knowledge. This is proposed as a direction for future investigation.
- Extending the Framework: The authors believe their framework can be used to generate quantitative predictions about feature sharing in the brain and to understand the interplay between distributed and modal representations. They also suggest extending the analysis to more complex multi-task scenarios and exploring how subtasks are learned from overarching goals.
Summary updated