Abstract
The inferotemporal cortex (ITC) is central to primate object recognition, yet its global functional organization remains partially understood. This study utilizes deep neural networks (DNNs) to construct a 25-dimensional space of visual features, mapping the spatial organization of feature preferences in both macaque and human brains using high-resolution fMRI. The resulting maps reveal a pair of orthogonal gradients that vary in spatial scale and highlight significant differences in functional organization between species. These findings provide a unified framework for understanding the topographic layout of high-level visual areas.
Introduction
The inferotemporal cortex (ITC) represents the pinnacle of the ventral visual stream, enabling the sophisticated object recognition capabilities of primates. While previous research has identified specialized "patches" for categories like faces, bodies, and places, these regions account for only about half of the monkey ITC. A comprehensive theory explaining the global organization of the temporal lobe—whether it is a collection of discrete modules or a continuous feature map—has remained elusive. Furthermore, the extent to which human and non-human primate "hardware" layouts correspond is not fully understood. This research seeks to define the high-dimensional coordinates the brain uses to navigate the visual world, moving beyond simple categorical labels.
Methodology
The researchers adopted a "model-first" strategy to map the brain's functional architecture. First, they extracted features by passing 200,000 images from the ImageNet dataset through AlexNet, a standard convolutional neural network. By applying Independent Component Analysis (ICA) to the penultimate layer (fc7), they defined a 25-dimensional (25D) space of independent visual features.
To map these features in the brain, fMRI was conducted on three macaque monkeys and four humans. Subjects were shown "positive" and "negative" images representing the extremes of each of the 25 feature axes to identify voxel-wise preferences. The resulting 25 maps were projected onto the cortical surface, allowing each ITC location to be treated as a 25D vector. Principal Component Analysis (PCA) was then used to identify the primary axes of spatial variation. Finally, Topographic Neural Networks (TNNs)—DNNs with spatial constraints—were used to model how such organization might emerge.
Key Insights and Results
The analysis revealed that the ITC's topographic map is highly complex, requiring at least 10 significant dimensions to describe accurately. This complexity far exceeds simpler 1D or 2D models based on animacy or size. A subset of seven interpretable features, such as spatial frequency and curvature, explained approximately 56% of neural variance, reaching 82.5% of the noise ceiling when supplemented with residual DNN features.
A major discovery was the "Orthogonal Gradient Principle": the brain organizes information via two overlapping spatial gradients. A coarse dorsal-to-ventral gradient (PC1) is overlaid by a finer, periodic gradient (PC2) running parallel to the Superior Temporal Sulcus (STS). Interestingly, while both species share a similar 25D representation ($r \approx 0.83$), the spatial layout is remarkably "crystalline" and ordered in monkeys, whereas it appears significantly more fragmented and distorted in humans. This suggests that human brain evolution, driven by the need to process artificial objects like tools and text, may have reconfigured the ancestral primate map.
Discussion and Future Directions
The study provides a powerful new lens for viewing the ITC, identifying previously unknown functional regions like the "inanimate stripes" in macaques. However, limitations remain. The use of static 2D images means that 3D cues (disparity) and dynamic information (motion) were not captured in the 25D space. Additionally, the linear mapping between DNN features and neural responses, while effective, leaves a portion of variance unexplained. Future research utilizing self-supervised models or ecologically diverse training tasks may further bridge this gap, offering deeper insights into the non-linear complexities of primate vision.
Summary updated