Abstract
Humans organize semantic knowledge into complex networks that encode relations between concepts. The structure of those networks has broad implications for human cognitive processes and theories of semantic development. While lexical networks derived from word associations suggest a "small-world" architecture, the extent to which this reflects environmental regularities is less understood. We analyzed semantic descriptors from four large datasets of naturalistic stimuli (visual, auditory, and audio-visual) to construct "grounded" semantic networks. Our results show these networks exhibit small-world properties with a degree distribution best captured by a truncated power law. These networks are predictive of human sensory judgments and reaction times, suggesting that the structure of semantic networks is grounded in the statistical regularities of the environment.
The Scale-Free Debate and Environmental Grounding
For decades, cognitive scientists have modeled human knowledge as a semantic network where concepts (nodes) are connected by relationships (edges). Traditionally, these networks are built using "lexical" data, such as word association tasks. While these lexical networks consistently show "small-world" properties—characterized by high clustering but short paths between any two points—two major questions remained: the validity of the "scale-free" property and the role of environmental grounding.
The "scale-free" property implies that the number of connections (degree) follows a power-law distribution, where a few "hubs" have massive connectivity. However, whether these networks are truly scale-free or if this is a statistical artifact remains debated. Furthermore, it was unclear to what extent these structures reflect the way the world actually looks and sounds, rather than just abstract linguistic usage.
Methods: Constructing Grounded Networks
The researchers constructed networks directly from human descriptions of naturalistic sensory stimuli across multiple modalities. They employed two primary methods to elicit semantic descriptors:
- STEP-Tag (Sequential Transmission Evaluation Pipeline): A collaborative "algorithm with people" where one participant proposes tags for a stimulus, and subsequent participants rate their relevance or add new ones. This iterative process filters noise and converges on high-quality consensus descriptors.
- Free Captions: As a control, participants provided open-ended descriptions.
Network Construction: Undirected, unweighted graphs were built where each unique word is a node. An edge is created between two words if they co-occur as descriptors for the same stimulus. To ensure statistical rigor, the study used maximum likelihood estimation and log-likelihood ratio tests to compare different distributional models (e.g., Power Law vs. Truncated Power Law).
Datasets
The study utilized four large-scale naturalistic datasets comprising 7,916 stimuli and over 610,000 human responses:
- BOLD5000: ~5,000 images of diverse indoor and outdoor scenes.
- Mini-Kinetics: 1,000 short video clips of everyday human activities.
- WikiArt: 1,000 modern and abstract paintings.
- Emotional Prosody (RAVDESS): 1,000 audio recordings of actors speaking with varying emotional tones.
Key Insights and Results
Small-World Topology
Multimodal networks exhibit the same small-world topology as lexical networks, with high clustering coefficients (>0.7) and short average path lengths (<3 steps). This suggests that the "search efficiency" of our mental maps is an adaptation to how features co-occur in nature.
The Truncated Power Law
The degree distribution is best captured by a truncated power law. While "hubs" exist (e.g., man, water, white), they are less hyper-connected than a pure power law predicts. This truncation likely stems from physical or cognitive constraints—there is a limit to how many distinct contexts a single concept can inhabit. In the BOLD5K and Emotional Prosody datasets, the truncated power law significantly outperformed all other candidate distributions ($p < 0.0001$).
Behavioral and Predictive Validity
These networks predict human behavior. A word's "degree" in these multimodal networks significantly predicts how fast a human can identify that word in a lexical decision task (Spearman $\rho$ between -0.15 and -0.42). Essentially, the more "connected" a word is in our sensory experience, the faster our brains process it. Furthermore, text embeddings derived from these tags predicted human similarity judgments with high accuracy.
Conclusion and Future Work
Our findings shed new light on the origins of semantic network structure by grounding it in environmental regularities. While the datasets are large, they remain thematically bounded. Future work should scale toward a "visual WordNet" and investigate cross-cultural regularities. Additionally, linking these semantic "hubs" to neural data (e.g., fMRI) could reveal how these structures are physically represented in the brain.
Summary updated