Inducing a lexicon of sociolinguistic variables from code-mixed text

Philippa Shoemark, James Kirby, Sharon GoldwaterEMNLP 2018

TL;DR

This paper introduces a novel method for automatically discovering sociolinguistic variables (like "football" vs. "fitba") from code-mixed social media text. By adapting techniques from gender-bias debiasing in word embeddings, they can efficiently generate ranked lists of potential linguistic variants, significantly aiding sociolinguistic research.

What Problem Does This Paper Address?

Studying language variation in large social media corpora is powerful, but a major hurdle is discovering the linguistic variables themselves. Researchers often rely on manual pairing of variants (e.g., "football" for British English and "fitba" for Scottish English), which is time-consuming and requires deep domain expertise. While previous work has focused on identifying dialectal regions or specific variant forms, this paper tackles the bottom-up discovery of equivalence pairs that represent sociolinguistic variation, like dialectal differences. The challenge is to find words that are semantically and functionally equivalent but differ in form, and to link these forms to their respective linguistic codes (e.g., British vs. Scottish English).

Methods

The core of the approach is to leverage word embeddings trained on code-mixed text. The authors adapt a framework originally used for debiasing gender in word embeddings. Here's a breakdown:

  1. Train Word Embeddings: They use the Skip-gram model to learn word embeddings from a large corpus of code-mixed text. These embeddings capture semantic relationships between words.
  2. Extract "Linguistic Code" Component: Using a small set of pre-defined "seed pairs" (e.g., (from, fae)), they compute a vector that represents the dimension of the embedding space aligned with the linguistic code difference (e.g., British vs. Scottish). This is done by averaging the differences between the embeddings of words in the seed pairs.
  3. Threshold Candidate Pairs: They generate all possible word pairs from the vocabulary and filter them based on a minimum cosine similarity threshold. This ensures that candidate pairs are semantically close. The threshold is automatically determined from the seed pairs.
  4. Rank Candidate Pairs: The crucial step involves ranking the remaining candidate pairs. They use a scoring function that measures how well the difference between a candidate word pair aligns with the extracted "linguistic code" vector. Pairs that align strongly are considered potential sociolinguistic variants.
  5. Filter Top-Ranked Pairs: To mitigate the effect of "hub" vectors (words that are nearest neighbors to many others, potentially leading to spurious similarities), they filter out pairs containing words that appear too frequently among the top-ranked candidates.

Datasets

The authors evaluate their method on three datasets derived from Twitter, representing different pairs of English language varieties:

  • BrEng/Scottish: A corpus of 9.4 million tweets from users geotagged in Scotland or discussing the 2014 Scottish Independence referendum. Seed pairs were curated from previous work.
  • BrEng/GenAm: An 8.3 million tweet corpus from US locations, combined with 1.7 million tweets from UK locations. Seed pairs were identified using the SAGE model and manual curation.
  • GenAm/AAVE: A dataset created by cross-referencing distinctively "American" terms with AAVE phonological processes, paired with their General American equivalents.

Key Insights and Ideas

  • Sociolinguistic Information is Encoded in Embeddings: The paper strongly suggests that word embeddings trained on code-mixed text implicitly capture sociolinguistic variation, not just semantic meaning. This is a significant finding for computational linguistics.
  • Adaptability of Embedding Techniques: The success in adapting a method designed for gender debiasing to discover dialectal variation highlights the generalizability of these embedding-based approaches for identifying various forms of linguistic bias and variation.
  • Efficiency Gain for Researchers: The method provides a ranked list of candidate pairs, drastically reducing the manual effort required for sociolinguistic research. Instead of sifting through vast amounts of text, researchers can focus on a much smaller, pre-filtered set.
  • Beyond Simple Spelling: While some pairs are simple spelling variations (e.g., color/colour), the method also identifies more nuanced alternations, suggesting it can capture deeper linguistic shifts.

Key Results

  • The system achieves precision@100 of over 70% for two of the three datasets (BrEng/Scottish and GenAm/AAVE).
  • Even with as few as five seed pairs, the method produces useful results, demonstrating robustness to the number and quality of initial seeds.
  • A naive baseline that only considers cosine similarity performs very poorly, underscoring the importance of the "linguistic code" component.
  • The BrEng/GenAm dataset showed lower precision, which the authors attribute to the smaller lexical differences between these standardized dialects, leading to more "grey areas" and cultural analogies in the top-ranked pairs.

Limitations and Future Work

  • BrEng/GenAm Nuances: The authors acknowledge that the BrEng/GenAm dataset presented challenges due to the subtle lexical differences, resulting in more culturally analogous pairs rather than strict linguistic variants.
  • Strict Evaluation Criteria: The evaluation criteria were strict, excluding pairs that might still be useful for some applications (e.g., those differing only in inflection or representing cultural equivalents).
  • Future Directions: The authors suggest that this framework could be extended to identify variation across genres or registers, and is expected to perform well with other language pairs that have significant vocabulary overlap or are frequently code-mixed.

Summary updated

BibTeX
@inproceedings{shoemark2018inducing,
  author = {Philippa Shoemark and James Kirby and Sharon Goldwater},
  title = {{Inducing a lexicon of sociolinguistic variables from code-mixed text}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2018},
}
Figure