TL;DR
This paper investigates why the human brain has specialized regions for processing certain categories like faces, using deep convolutional neural networks (CNNs) as computational models. Surprisingly, even a single CNN trained to do both face and object recognition simultaneously performed nearly as well as separate networks, suggesting that networks might discover functional segregation internally.
What Problem Does This Paper Address?
The human brain exhibits remarkable functional specialization, with distinct cortical areas dedicated to processing specific types of information, such as faces (e.g., the Fusiform Face Area) and scenes. This raises fundamental questions: why is this specialization advantageous for brain design, and why are some categories processed this way while others are not? To explore these questions, the authors use deep convolutional neural networks (CNNs), which have proven effective at modeling visual processing in humans and animals, to test hypotheses about the necessity of explicit functional segregation for tasks like face and object recognition.
Methods
The researchers employed a methodology inspired by previous work on auditory processing tasks. They trained CNNs (specifically, VGG16 architecture) in three main configurations:
- Separate Networks: Two independent CNNs were trained, one exclusively for face identity recognition and the other solely for object categorization. This establishes a baseline performance for each task when the network is free to learn task-specific features without constraints.
- Fully Shared Dual-Task Network: A single CNN was trained to perform both face and object recognition simultaneously. This network shared all its layers.
- Branched Dual-Task Networks: A series of networks were trained on both tasks, but with progressively earlier "branching points" where the network would split into task-specific pathways after certain layers. This allowed them to investigate how much of the network could be shared before performance degraded.
To assess feature differences, they used a "decoding" approach: they extracted activations from the penultimate layer of each network and trained a support vector machine to classify novel object categories or face identities. This helps determine if the features learned by the face-only network are useful for object recognition, and vice-versa.
Datasets
- Face Identity Recognition: A randomly sampled set of 1,000 identities (500 female) from the VGGFace2 database was used, with 300 images per identity for training and 50 for validation.
- Object Categorization: 600 randomly sampled categories from the ILSVRC-2012 database were used, with 500 images per category for training and 50 for validation.
Key Insights and Ideas
- Distinct Feature Spaces: Networks trained exclusively on faces or objects learn distinct feature representations. This is evidenced by the fact that features from a face-trained network are poor at decoding object categories, and vice-versa.
- Surprising Robustness of Shared Networks: Contrary to the intuition that explicit segregation would be necessary, a single CNN trained on both face and object recognition simultaneously performed nearly as well as two separate networks. This suggests that networks might be capable of developing internal, "hidden" functional segregation even without explicit architectural constraints.
- Early vs. Late Segregation: While performance didn't drop drastically even with a fully shared network, the authors observed that performance did begin to decline when branching occurred very early (after the first few pooling layers). This hints that some degree of early specialization might be beneficial, but the network can tolerate significant sharing.
Key Results
- Decoding Accuracy:
- Face CNN: 50% correct for object decoding (chance 10%), 100% correct for face decoding.
- Object CNN: 65% correct for face decoding (chance 10%), 88% correct for object decoding.
- Dual-Task Performance:
- Separate networks achieved high performance (Face: Top-1 94%, Top-5 98%; Object: Top-1 57%, Top-5 81%).
- The fully shared dual-task network performed nearly as well as the separate networks.
- Even networks with branching points after the fifth pooling layer showed only a small performance drop (e.g., object branch after pool 5: 1.49% drop; face branch after pool 5: 0.53% drop).
- The network that did not branch until the very last layer performed almost identically to the separate networks.
Limitations and Future Work
- Disambiguating Hidden Specialization: The authors acknowledge that their current results are consistent with two hypotheses: either face and object recognition can be performed well by a shared pool of common features, or the shared network has learned "hidden" functional specialization. They are actively working to disambiguate these two possibilities.
- Investigating Feature Space Nature: Ongoing analyses are focused on understanding the nature of these feature spaces at each layer, how early they diverge, and how they differ.
- Comparing with Other Tasks: They plan to extend their comparison to other task pairs, such as cars and objects, for which functional specialization is not as clearly observed in the human visual system, to see if similar results emerge.
Summary updated