TL;DR
This paper presents a novel approach to modeling how word meanings change over time (Lexical Semantic Change). Their method, Outlier2Cluster, combines Word Sense Disambiguation (WSD) and Word Sense Induction (WSI) models with a new Novel Sense Detection (NSD) component to achieve state-of-the-art results on the AXOLOTL-24 shared task. This work highlights the importance of robustly identifying truly new word senses for accurate semantic change modeling.
What Problem Does This Paper Address?
Understanding how word meanings evolve is crucial for historical linguistics, digital humanities, and even tracking the evolution of language in real-time. Traditional approaches to Lexical Semantic Change (LSC) often focused on identifying if a word's meaning changed or ranking potential changes. However, the AXOLOTL-24 shared task introduced a more granular challenge: for a given word, automatically assign each individual usage in a newer time period to a defined sense from an older period, or to a newly discovered sense. This requires not only disambiguating known senses but also detecting and accounting for entirely novel meanings that emerge over time. Existing methods often struggle with this dual challenge, either failing to detect novel senses or misattributing new usages to existing, but incorrect, definitions.
Methods
The authors propose three main methods, with their best performing model, Outlier2Cluster, being a sophisticated orchestration of existing techniques and a novel component:
- Word Sense Disambiguation (WSD): Models like GlossReader are used to assign a new word usage to one of the predefined "old senses" based on its definition.
- Word Sense Induction (WSI): Models like Agglomerative Clustering group new word usages into clusters, aiming to discover latent senses without relying on predefined definitions.
- Novel Sense Detection (NSD): This is the authors' key contribution. They frame NSD as an outlier detection problem. A logistic regression classifier is trained to identify usages that are too "distant" from all provided old sense definitions. This model uses features derived from GlossReader's representations of usages and glosses, along with auxiliary features like the number of old/new usages.
Outlier2Cluster works by first independently running WSD and WSI. Then, the NSD model identifies potential "outlier" usages (likely representing new senses). These outlier usages are then assigned to clusters identified by the WSI model, while non-outlier usages are assigned the sense predicted by the WSD model. This allows the system to leverage both known senses and discover new ones.
Datasets
The research was conducted on datasets from the AXOLOTL-24 shared task, specifically for Finnish, Russian, and German.
- Finnish: Provided train and development sets. The data often features older characters and uses modern Finnish glosses, creating a potential mismatch.
- Russian: Provided train and development sets. Notably, Russian datasets sometimes lack old usages for existing senses, requiring methods that don't strictly rely on them.
- German: Provided only a test set, requiring a "zero-shot" evaluation for the NSD component.
Key Insights and Ideas
- The importance of Novel Sense Detection: The paper strongly suggests that accurately identifying truly novel senses is critical for high-quality semantic change modeling, especially for languages like Russian where senses often diverge significantly.
- WSD vs. WSI trade-offs: Pure WSD methods excel at tasks where new and old senses overlap heavily (like Finnish), achieving high ARI. However, they fail to detect new senses. WSI methods are better at discovering new senses but may not align perfectly with existing definitions, impacting ARI.
- NSD as outlier detection: Framing novel sense detection as an outlier problem, where usages are "distant" from known senses, proved effective.
- Feature importance varies by language: Features that help NSD models perform well on Finnish (like distance-based features) are less effective on Russian, where auxiliary features (like the number of usages) become more important.
Key Results
The authors' Outlier2Cluster method achieved state-of-the-art (SOTA) results on the AXOLOTL-24 shared task for the first subtask, outperforming other participants across multiple metrics.
- Overall Performance: Outlier2Cluster achieved SOTA results on average across languages.
- Finnish vs. Russian: Pure WSD methods achieved the highest ARI on Finnish, while Outlier2Cluster (with its NSD component) performed best on Russian, demonstrating its ability to handle languages with more divergent semantic change.
- NSD Model Effectiveness: The NSD model, particularly when trained with appropriate features for each language, significantly improved the overall semantic change modeling performance, especially in scenarios with disjointed senses.
Limitations and Future Work
- Hyperparameter Sensitivity: The performance of Outlier2Cluster is sensitive to the threshold used in the NSD model to distinguish outliers from normal usages.
- Language Specificity: The NSD model needs to be trained separately for each language, and its effectiveness can vary.
- Historical Language Handling: The underlying GlossReader model is based on modern language models (XLM-R) and may not be optimally suited for significantly older or historically divergent language forms. Future work could explore dataset-specific fine-tuning for such cases.
- Further NSD Improvement: The authors suggest that improving the NSD model is a promising direction for future research, as it directly addresses the challenge of detecting genuinely new word senses.
Summary updated