Abstract
Data-hungry neuro-AI modelling requires ever larger neuroimaging datasets. CNeuroMod-THINGS meets this need by capturing neural representations for a wide set of semantic concepts using well-characterized images in a new densely-sampled, large-scale fMRI dataset. Importantly, CNeuroMod-THINGS exploits synergies between two existing projects: the THINGS initiative (THINGS) and the Courtois Project on Neural Modelling (CNeuroMod). For CNeuroMod-THINGS, four participants each completed 33-36 sessions of a continuous recognition paradigm using approximately 4000 images from the THINGS stimulus set spanning 720 categories. We report behavioural and neuroimaging metrics that showcase the quality of the data. By bridging together large existing resources, CNeuroMod-THINGS expands our capacity to model broad slices of the human visual experience.
Introduction
In the burgeoning field of "Neuro-AI," researchers aim to build computational models that mimic human vision. However, these models are data-hungry. Existing fMRI datasets often face a trade-off: they are either "wide" (showing many images to many people, but with few repetitions) or "deep" (showing a few images many times to a few people). Wide datasets lack the signal-to-noise ratio needed to analyze responses to individual items, while deep datasets often lack the semantic variety needed to understand how the brain categorizes the world.
CNeuroMod-THINGS bridges this gap. It leverages the THINGS initiative—which provides a massive, annotated library of natural objects—and the Courtois Project on Neural Modelling (CNeuroMod)—a "deep phenotyping" project that scans the same individuals for hundreds of hours—to create a dataset that is both semantically broad and statistically robust.
Experimental Design and Methods
The researchers employed a "deep phenotyping" approach, focusing on four healthy participants. Each participant completed between 33 and 36 fMRI sessions.
The Continuous Recognition Task
Participants performed a "continuous recognition task" while viewing a stream of images. They used a game controller to indicate whether they had seen the image before and their level of confidence. Each participant saw approximately 4,300 unique images from 720 object categories. Crucially, every image was repeated three times—either within the same session or across different sessions—to allow for high-reliability analysis.
Denoising and Estimation
To extract the signal for individual images, the team used GLMsingle, a specialized tool that uses data-driven techniques to optimize "beta" estimates for every single trial. The dataset also includes high-resolution structural scans, retinotopy, and functional localizers to ensure results are mapped accurately to each individual's brain anatomy.
Key Insights and Results
Data Quality and Signal Robustness
Participants maintained high engagement, with a median response rate of >98% and high accuracy ($d'$ scores between 1.5 and 1.9). By repeating every image three times, the researchers calculated "noise ceilings"—a measure of how much of the brain's signal is driven by the image versus random noise. In early visual areas (V1-V3) and category-selective regions (FFA, PPA, EBA), noise ceilings reached over 70%, confirming a strong link between fMRI signal and visual content.
Semantic Organization and Memory
Using t-SNE dimensionality reduction, the authors showed that brain activity in specific regions naturally clusters by category. For example, in the Fusiform Face Area (FFA), the brain's response to a dog's face is more similar to a human face than to a toaster, even at the single-trial level. Additionally, the dataset captures "repetition suppression," where the brain's response to an image decreases once it becomes familiar, allowing for the study of the intersection between vision and memory.
Discussion and Conclusion
While the small sample size (four participants) optimizes the dataset for within-subject modeling rather than population generalization, the depth of data per subject is unprecedented. One participant (sub-01) reported bouts of drowsiness, but data quality remained sufficient for inclusion.
CNeuroMod-THINGS expands our capacity to model broad slices of the human visual experience. The entire dataset is available via DataLad, encouraging the community to use it for training new neuro-inspired AI models and integrating it with other naturalistic CNeuroMod datasets, such as movie-watching and video-game playing.
Summary updated