Discovering Phonesthemes with Sparse Regularization

Nelson F. Liu, Gina-Anne Levow, Noah A. SmithNAACL 2018

TL;DR

This paper introduces a novel computational method to automatically discover "phonesthemes" – non-arbitrary sound-meaning connections in language – by treating it as a feature selection problem. Their approach successfully identifies known phonesthemes and uncovers new ones, validated by human judgments, offering a data-driven way to explore the iconic aspects of language.

What Problem Does This Paper Address?

Language is often characterized by arbitrariness, meaning the form of a word (its sound or spelling) has no inherent connection to its meaning. However, linguists have long observed exceptions to this rule, such as iconicity, where word form does resemble meaning. Phonesthemes are a prime example: submorphemic sound units that consistently appear in words with similar meanings (e.g., the initial "gl-" in words related to light like "gleam," "glitter"). While many phonesthemes have been proposed and studied, there's a lack of robust methods for automatically discovering them from large datasets. This paper aims to fill that gap by developing a computational model that can identify these non-arbitrary sound-meaning links.

Methods

The core of the paper's methodology lies in framing phonestheme discovery as a feature selection problem. They train a linear regression model to predict word meaning (represented as semantic vectors) using submorphemic units (specifically, initial phoneme bigrams) as features. To automatically select the most predictive submorphemes, they employ sparse regularization, specifically the elastic net. This technique encourages the model to assign zero weight to irrelevant features, effectively highlighting the submorphemes that are most strongly associated with word meaning.

A key innovation is their two-stage approach to mitigate the influence of morphemes. Since morphemes (meaningful units like prefixes and suffixes) can also carry meaning and influence word vectors, they could be mistaken for phonesthemes. To address this, they first train a model to predict word vectors from morpheme-level features. Then, they use the residuals (the unexplained variance) from this morpheme model as the target for their phonestheme extraction model. This effectively "removes" the meaning already explained by morphology, allowing the second model to focus on non-morphemic, sound-meaning relationships.

Datasets

  • GloVe word embeddings: 300-dimensional GloVe embeddings trained on the cased Common Crawl corpus were used as the semantic representations of words.
  • CELEX lexical database: Used for morphological analysis to identify morphemes for the two-stage mitigation approach.
  • CMU Pronouncing Dictionary: Used to phonemicize words, converting orthographic forms into their ARPAbet symbol sequences, which are crucial for analyzing phonetic features.

The authors filtered the initial GloVe embeddings to retain 6633 word vectors, removing non-English words, named entities, and very rare or common words to focus on a cleaner set of English words.

Key Insights and Ideas

  • Phonesthemes as a Feature Selection Problem: The paper demonstrates that a sparse linear regression model can effectively identify phonesthemes by treating submorphemic units as features and word meaning as the target.
  • Two-Stage Morpheme Mitigation: The proposed method for separating morphemic from phonesthetic contributions is a significant step towards more accurate phonestheme discovery, as it directly tackles a major confounding factor.
  • Data-Driven Discovery: The approach moves beyond relying solely on linguists' intuitions, offering a systematic way to uncover potential phonesthemes from large-scale linguistic data.
  • Validation of Non-Arbitrary Links: The success in identifying known phonesthemes and the positive human judgments suggest that these sound-meaning connections are not just linguistic curiosities but may have a basis in human cognition.

Key Results

  • The model identified 123 candidate phonesthemes from a hypothesis space of 307 initial phoneme bigrams.
  • 13 out of the top 15 model-predicted phonesthemes were found in Hutchins' (1998) compilation of proposed phonesthemes.
  • Human annotators rated words containing model-selected phonesthemes as fitting their meaning better (average rating of 3.66) compared to words with unselected phonestheme candidates (average rating of 3.08). This difference was statistically significant (p < 10⁻⁹).
  • The study identified two new potential phonesthemes, "br-" and "wi-", which seem to evoke concepts of "raw force" and "fragility," respectively, and were validated by human judgments.

Limitations and Future Work

  • Focus on Initial Bigrams: The current model is limited to word-initial phoneme bigrams. The authors acknowledge that phonesthemes can occur in other positions (e.g., word-final) and at different lengths (e.g., trigrams).
  • Language Specificity: The current work focuses on English. Extending the method to other languages is a natural next step.
  • Exploration of Other Positions/Lengths: Future work could explore phonesthemes at the end of words, within words, or using longer n-grams.
  • Deeper Psychological Validation: While human judgments were positive, further exploration into the psychological reality of newly discovered phonesthemes like "br-" and "wi-" is suggested.
  • Orthographic vs. Phonemicized Vectors: The paper notes that the model can be applied to either orthographic or phonemicized vectors, but ideal modeling is at the phoneme level. Future work could explore the trade-offs more deeply.

Summary updated

BibTeX
@inproceedings{liu2018discovering,
  author = {Nelson F. Liu and Gina-Anne Levow and Noah A. Smith},
  title = {{Discovering Phonesthemes with Sparse Regularization}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2018},
}
Figure