Whether you see a picture of a cat, hear the word "cat," or read the letters C-A-T, your brain eventually arrives at the same high-level concept. This phenomenon—where neural activity converges on a shared meaning regardless of the sensory input—is known as modality invariance.
Recent research across cognitive neuroscience and artificial intelligence suggests that both biological and artificial systems do not just process sensory data in isolation. Instead, they map diverse inputs into a centralized, amodal "semantic hub" where the core meaning of the world resides.
The Semantic Hub Hypothesis
In the human brain, sensory processing begins in specialized regions: the visual cortex for sight and the auditory cortex for sound. However, as information moves deeper into the brain, these modality-specific details are stripped away.
Timothy T. Rogers and colleagues Rogers et al. (2006) identified the anterior temporal lobes (ATLs) as a critical "hub" for this integration. While damage to sensory areas might cause blindness or deafness, damage to the ATL leads to semantic dementia—a loss of the concept itself. A patient might see a robin but fail to understand it is a "bird," regardless of whether they are looking at it or hearing its song.
This convergence is remarkably precise. Fatma Deniz and Jack L. Gallant demonstrated in Deniz et al. (2019) that the semantic maps created when a person listens to a story are nearly identical to the maps created when they read the same story. Once the initial sensory "wrapper" is removed, the brain’s internal representation of the narrative is amodal.
Decoding Meaning Across Views
If the brain truly uses a shared code for concepts, we should be able to "read" a person's thoughts about an object even if we change how they perceive it. This is the logic behind cross-modal decoding.
Irina Simanova and her team Simanova et al. (2010) explored whether a classifier trained on EEG data from visual stimuli could identify the same concept when presented as a spoken word. Similarly, SUBBA REDDY OOTA showed in Oota et al. (2022) that Transformer-based models can perform "zero-shot cross-modal transfer," predicting brain activity for images using only text-trained decoders.
This suggests that the brain maintains a shared representational geometry. As Shurui Li and Yuanning Li argue in Li et al. (2025), the brain maps distinct modalities into a common embedding space to facilitate rapid retrieval and comparison, allowing linguistic expectations to reshape visual perception in real-time.
The Emergence of Platonic Spaces in AI
The drive toward modality invariance isn't unique to biology; it appears to be a fundamental property of high-performing computational models.
In the world of Vision-Language Models (VLMs), researchers have observed a "Platonic" convergence. Zoe Wanying He and Sean Trott found that models trained on entirely disjoint data (only images vs. only text) spontaneously develop aligned representational spaces He & Trott (2025). This suggests that there is an underlying "structure of reality" that both humans and AI models eventually discover.
This convergence allows for architectural efficiencies. For example, the DeepInsert method by Akhil Bhimaraju and colleagues Bhimaraju et al. (2025) skips early processing layers for non-text tokens, inserting them directly into the deeper layers of a transformer where the semantic space has already become modality-neutral.
Why Invariance Works: Concept Detectors
Why does the system bother to abstract away the sensory details? The goal is to create invariant concept detectors.
In neuroscience, these are famously called "Halle Berry neurons"—single units in the medial temporal lobe that fire for a specific person whether you see their face, their name, or a caricature Blumberg & Kreiman (2010). Darryl Hannan and Garrett T. Kenyon demonstrated that these units can emerge computationally through Deep Sparse Coding, which uses lateral inhibition to force the network to find the most efficient, abstract representation of an input Hannan & Kenyon (2017).
In AI, Stefanie Jegelka and Phillip Isola showed that simply forcing different modalities to share parameters causes the model to develop individual units that respond to specific concepts (like "towers" or "beds") regardless of the input format Jegelka & Isola (2025).
Conclusion
The evidence from both fMRI studies and large-scale neural networks points to a singular conclusion: intelligence relies on the ability to transcend the "here and now" of sensory data. By mapping pixels, sound waves, and text into a shared, modality-invariant space, neural systems—whether biological or artificial—can build a stable, conceptual understanding of the world that remains consistent across every way we experience it.