Improving Automated Prediction of English Lexical Blends Through the Use of Observable Linguistic Features

Jarem SaundersACL 2023

TL;DR

This paper introduces a novel machine learning architecture for predicting English lexical blends, achieving significantly higher accuracy than previous state-of-the-art models. By leveraging observable linguistic features rather than solely relying on deep learning's "black box" latent spaces, the authors demonstrate a more interpretable and effective approach to this challenging linguistic phenomenon.

What Problem Does This Paper Address?

Lexical blending, the process of combining parts of two words to create a new one (e.g., "brunch" from "breakfast" and "lunch"), is notoriously difficult to predict automatically. While previous attempts using deep neural networks like LSTMs have shown some success, they often struggle to achieve high prediction rates and can produce phonotactically invalid outputs. The core challenge lies in capturing the complex, often subtle, linguistic factors that govern how words are combined and truncated. This paper aims to improve the accuracy and interpretability of automated lexical blend prediction by exploring the utility of observable linguistic features.

Methods

The authors propose a novel model architecture that combines several key components:

  • Blend Candidate Generation: The model iteratively generates plausible blend candidates by taking prefixes from the first input word and suffixes from the second, using grapheme-phoneme alignments. This "exhaustive generation" strategy ensures a wide range of potential blends are considered.
  • Observable Linguistic Feature Extraction: For each candidate blend, a rich set of linguistically-motivated features are extracted. These features are based on descriptive characteristics of blends found in linguistic literature, such as syllable boundaries, stress patterns, and the proportion of segments preserved from each input word.
  • Probability Calculation: Polynomial regression and Random Forest classifiers are used to learn the probability of each candidate being a valid blend, based on the extracted features. These models were chosen for their interpretability compared to deep neural networks.
  • Candidate Selection: The blend candidate with the highest predicted probability is selected as the model's output.

The novelty lies in the explicit use of observable linguistic features as the primary input for the prediction models, moving away from reliance on latent representations learned by deep neural networks.

Datasets

The study utilizes three distinct corpora of lexical blends:

  • Deri & Knight (2015) Corpus: Acquired from online resources like Wikipedia, Wiktionary, and Urban Dictionary. 322 blends were used.
  • Gangal et. al. (2017) Corpus: Also sourced from online resources. 1092 blends were used.
  • Shaw (2014) Corpus: A curated collection from an earlier dictionary. 1096 blends were used.

These datasets were used for training and testing, and the model's performance was compared against previous benchmarks established on these or similar datasets.

Key Insights and Ideas

  • Observable Features Trump Latent Spaces: The most significant insight is that explicitly defined, observable linguistic features can lead to superior performance in blend prediction compared to models relying solely on latent representations learned by deep neural networks. This suggests that interpretability and linguistic grounding can be powerful tools in NLP.
  • Interpretability Matters: The use of regression and tree-based models allows for examination of feature weights, offering insights into why certain blends are predicted, which is a valuable contrast to the "black box" nature of deep learning models.
  • Potential for Other Linguistic Phenomena: The success of this approach suggests it could be a fruitful framework for tackling other unpredictable or infrequently attested linguistic processes.

Key Results

  • Improved Accuracy: The proposed model consistently outperformed previous benchmarks across all three corpora. For instance, on the Shaw corpus, the best-performing model achieved an average of 74.13% correct predictions, a significant improvement over the previous benchmark of 48.75% on a similar dataset.
  • Polynomial Regression Dominance: Polynomial regression models, particularly when using the full feature set, consistently yielded the highest prediction rates across all corpora.
  • Qualitative Findings: Error analysis revealed that the model tends to over-preserve phonemic material from input words rather than over-deleting, leading to a tendency for predicted blends to be longer than the desired output.

Limitations and Future Work

  • Data Dependency: The model requires access to grapheme-phoneme alignment information for all input words, which limits its applicability to languages with less readily available linguistic data.
  • Bias Towards Longer Blends: Qualitative analysis indicated a tendency for the model to produce longer blends, suggesting a need for further research into measures to reduce this bias.
  • Future Directions: The authors suggest further work on hyperparameter tuning, optimizing the feature set, and exploring measures to mitigate the observed bias. They also propose using the model's interpretability to test hypotheses about blend generation and compare its performance against large language models using genuinely held-out data.

Summary updated

BibTeX
@inproceedings{saunders2023improving,
  author = {Jarem Saunders},
  title = {{Improving Automated Prediction of English Lexical Blends Through the Use of Observable Linguistic Features}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2023},
}
Figure