Abstract
Psycholinguistic properties such as imageability and concreteness are essential for understanding human language processing, yet they traditionally rely on labor-intensive human annotations. This paper introduces the Neighborhood Stability Measure (NSM), a novel unsupervised method that estimates these properties by analyzing the geometric structure of text embedding spaces. By quantifying the "peakedness" of a word's neighborhood within a large-scale text collection, NSM effectively captures visual-semantic traits without the need for multimodal data or expensive generative models. Our results demonstrate that NSM significantly outperforms existing unsupervised baselines, achieving high correlation with human ratings and providing 100% vocabulary coverage.
Introduction
In psycholinguistics, imageability (the ease with which a word evokes a mental image) and concreteness (the degree to which a word refers to perceptible entities) are fundamental to understanding how humans acquire and process language. Historically, measuring these traits has required extensive human surveys, such as the MRC Psycholinguistic Database.
While computational approaches have attempted to automate this process, they often face significant hurdles. Multimodal methods require scarce paired image-caption data, while generative approaches—such as those utilizing DALL-E—are computationally prohibitive for large vocabularies. Furthermore, many lexical-based methods suffer from low coverage due to vocabulary mismatch. This research addresses these challenges by asking: Can we extract visual-semantic properties using only the distributional structure of text embeddings?
The Neighborhood Stability Measure (NSM)
The authors propose that the geometric distribution of words in a vector space reflects their psychological properties. Specifically, imageable words (e.g., "beach," "banana") tend to appear in specific, constrained contexts, creating "sharp peaks" or stable clusters in the embedding space. Conversely, abstract words (e.g., "gratitude") appear in diverse contexts, leading to "flatter," less stable distributions.
The Algorithm
To quantify this stability, the NSM algorithm follows four primary steps:
- Embedding: A large text collection (e.g., image captions) is embedded using a standard model like
all-MiniLM-L6-v2. - Neighbor Identification: For a query word $q$, the system identifies its $k$ nearest neighbors within the collection.
- Stability Check: For every neighbor $u$ in the set, the algorithm checks if $u$’s own nearest neighbor also resides within the original neighborhood of $q$.
- Scoring: The NSM score is the fraction of neighbors that satisfy this mutual proximity. A score near 1 indicates a highly stable, isolated cluster, which correlates with high imageability.
Experimental Setup
The method was evaluated against human ground truth from the MRC Psycholinguistic Database (4,848 words) and the Brysbaert et al. dataset (37,058 words). To test the impact of the source text, the authors utilized several corpora:
- Visual-Centric Text: MS COCO (1.5M captions), CC3M (3.3M), and CC12M (12M).
- General Text: QUORA, FIQA, and MS MARCO.
Key Insights
Geometry as a Proxy for Perception
The most significant finding is that the mathematical stability of a neighborhood in a vector space serves as a robust proxy for human psychological traits. This suggests that the "visual" nature of a word is encoded in its linguistic distribution, provided the text is rich enough.
The "Visual Region" of Language
NSM performance was markedly higher on image caption datasets than on general text corpora. This indicates that visual-semantic properties are best captured in language specifically intended to describe visual phenomena. As the size of the caption dataset increased from 1.5M to 12M, the correlation with human ratings improved, suggesting that larger datasets provide a more granular sampling of the semantic space.
The Role of Dimensionality
Counter-intuitively, lower-dimensional embedding models (e.g., 384-d) often outperformed higher-dimensional ones (e.g., 1024-d). In very high-dimensional spaces, pairwise distances tend to concentrate, a phenomenon known as the "curse of dimensionality," which can obscure the local neighborhood structures that NSM relies upon.
Results and Performance
- Correlation: NSM achieved a Spearman correlation of 0.66 for imageability and 0.58 for concreteness using the CC12M dataset, outperforming previous unsupervised baselines.
- Coverage and Efficiency: Unlike lexical methods that often cover less than 15% of target vocabularies, NSM provides 100% coverage for any word that can be embedded. Furthermore, while generative methods can take hundreds of hours to compute, NSM relies on fast Approximate Nearest Neighbor (ANN) searches, completing tasks in seconds or minutes.
- Classification: In binary classification tasks (distinguishing concrete from abstract words), NSM demonstrated high robust predictive power, maintaining superior AUC scores across various thresholds.
Conclusion
NSM demonstrates that the distributional structure of text embeddings contains deep psycholinguistic information that can be extracted without multimodal or generative overhead. While currently limited to English and single-word tokens, future work may explore the compositionality of these properties at the sentence level and their application to other languages and psychological dimensions, such as emotional valence.
Summary updated