Abstract
Independent Component Analysis (ICA) is an algorithm originally developed for finding separate sources in a mixed signal, such as a recording of multiple people in the same room speaking at the same time. It has also been used to find linguistic features in distributional representations. In this paper, we used ICA to analyze words embeddings. We have found that ICA can be used to find semantic features of the words and these features can easily be combined to search for words that satisfy the combination. We show that only some of the independent components represent such features, but those that do are stable with regard to random initialization of the algorithm.
Introduction: The Black Box of Word Embeddings
Word embeddings, such as word2vec, serve as the foundational architecture for modern Natural Language Processing (NLP). By mapping words into high-dimensional vector spaces, these models capture complex semantic relationships. However, these embeddings are notoriously opaque; while the distance between vectors is meaningful, the individual dimensions themselves are typically uninterpretable to humans.
Standard dimensionality reduction techniques like Principal Component Analysis (PCA) often fail to clarify these representations. PCA creates a hierarchy where the primary components attempt to explain maximum variance, resulting in "busy" dimensions that conflate multiple broad categories (e.g., mixing syntax with broad semantics). This research explores Independent Component Analysis (ICA) as a method to decompose dense embeddings into "atomic," sparse, and highly interpretable semantic features.
Methodology: ICA and the Word Intruder Test
The researchers applied ICA to 512-dimensional word2vec vectors. Unlike PCA, which seeks orthogonal directions, ICA searches for independent sources by maximizing the non-Gaussianity of the data. This process identifies specific directions in the vector space that correspond to distinct semantic themes, such as "animals," "legal terms," or "names."
To quantify interpretability, the authors employed the word intruder test. In this setup, a subject is presented with five words: four "extreme words" (those for which a specific component is a dominant feature) and one "intruder" word from a different component. If a component is truly interpretable, a human or model should easily identify the outlier.
Uni-directionality and Semantic Features
A key discovery in this work is that interpretable components are often uni-directional. In many cases, a component represents a clear semantic feature in one direction (e.g., positive values), while the opposite direction contains only random noise.
Key Results: Scaling Interpretability with LLMs
The study compared the interpretability of raw word2vec dimensions against ICA-transformed components. Human trials revealed that ICA components were significantly more meaningful, with subjects correctly identifying intruders 42% of the time, compared to only 31% for raw dimensions.
To address the "evaluation bottleneck" of human labor, the researchers tested whether Large Language Models (LLMs) could serve as proxies for human judgment. The results were striking: GPT-4 not only performed the task reliably but significantly outperformed human annotators.
As shown in Table 1, GPT-4 identified the intruder in ICA components 524 times (out of 512 dimensions tested in both directions), nearly doubling its success rate compared to raw word2vec dimensions (273). This suggests that LLMs can provide a fast, inexpensive, and scalable way to quantify model interpretability.
Discussion: Semantic Algebra and Bias Mitigation
Because ICA components represent independent features, they can be manipulated using simple "semantic algebra." By multiplying different components, researchers can isolate words that satisfy multiple criteria—for instance, combining a "sound" component with an "animal" component to find words like barking or chirping.
Furthermore, ICA provides a lens through which to view latent social biases. In datasets like Wikipedia, general "people" components often split into gendered directions. By identifying these specific axes of bias, ICA offers a potential pathway for researchers to "scrub" or mitigate harmful social biases from AI models without destroying the underlying semantic utility of the embeddings.
Conclusion
Independent Component Analysis successfully decomposes the "black box" of word embeddings into stable, interpretable semantic units. While roughly 30% of components remain non-interpretable noise, the remaining features provide a clear map of the linguistic "DNA" within a model. By leveraging LLMs for automated evaluation, this framework allows for the rapid assessment and refinement of more transparent and fair NLP systems.
Summary updated