Minimalist Entity Disambiguation for Mid-Resource Languages

Benno KruitACL 2023

TL;DR

This paper introduces MiniNED, a Python library for creating small, efficient Named Entity Disambiguation (NED) models for mid-resource languages. It demonstrates that these minimalist models can achieve competitive performance with significantly fewer computational resources than state-of-the-art systems, making NED more accessible for practical applications.

What Problem Does This Paper Address?

Named Entity Disambiguation (NED) is a crucial task in Natural Language Processing (NLP) that involves linking mentions of entities in text to their corresponding entries in a knowledge base, like Wikipedia. While powerful NED systems exist for well-resourced languages like English, there's a significant gap for "mid-resource" languages (e.g., Persian, Japanese, Tamil). This gap arises because state-of-the-art NED models are often very large and computationally expensive to train and deploy, making them impractical for many real-world scenarios, especially in resource-constrained environments. The authors aim to bridge this gap by developing a framework for creating smaller, more manageable NED models that still perform well.

Methods

The core of the paper's approach is the development of MiniNED, a framework for building minimalist NED models. Instead of relying on massive neural language models, MiniNED uses simpler models: multinomial logistic regression classifiers trained with hashed Bag-of-Word features. These models are trained using Vowpal Wabbit, a fast online learning system.

Key methodological aspects include:

  • Candidate Generation: Entities are identified by filtering Wikipedia data using heuristics that consider surface form appearance on disambiguation pages, string similarity to entity labels, and the entropy of prior probability distributions.
  • Candidate Filtering: To reduce computational load and improve efficiency, candidate sets are filtered by selecting a percentage (quantile) of the most frequently observed mention-entity instances.
  • Model Size Control: The size of the models is controlled by the number of bits used for feature hashing, which also acts as a regularization mechanism to prevent overfitting. This allows for a trade-off between model size and accuracy.
  • Stemming: The impact of stemming (removing word inflections) on ambiguity and performance is analyzed across different languages.

Datasets

The primary dataset used for training and evaluation is Wikipedia dumps (specifically, dumps from 2022-03-01). The authors also leverage the Mewsli-9 benchmark dataset, which consists of entity mentions from nine languages. For their experiments, they replace the English portion of Mewsli-9 with Dutch, aligning with their focus on mid-resource languages. They also mention modifying Mewsli-9 by removing links to disambiguation and list pages.

Key Insights and Ideas

  • Narrow Performance Range: For many mid-resource languages, the performance gap between a simple baseline (always picking the most frequent entity) and a near-perfect model is surprisingly small. This suggests that simpler models can indeed achieve acceptable performance.
  • Model Size vs. Performance Trade-off: Performance generally improves with model size, but with diminishing returns. Models significantly smaller than state-of-the-art can achieve reasonable results, especially when optimized.
  • Impact of Stemming: Stemming can increase entity name ambiguity but also decrease the number of unseen mentions. Its effect varies significantly across languages due to morphological differences.
  • Data Noise: Wikipedia hyperlink data is inherently noisy (e.g., links to disambiguation pages, incorrect links). For smaller datasets, this noise becomes a more significant problem, highlighting the need for simple models that can be tuned to mitigate it.
  • Shadowed Mentions: A key challenge is disambiguating "shadowed" entity mentions, which share a surface form with more popular entities. This requires careful attention to training data and model assumptions.

Key Results

  • MiniNED models, which are a fraction of the size of recent state-of-the-art systems, achieve reasonable NED performance across nine languages.
  • The framework allows practitioners to tune model complexity and adjust for specific use-cases while maintaining performance.
  • Figure 4 shows that performance improves logistically with model size, with significant gains achieved with models under a few hundred MB.
  • Table 1 presents micro F1-scores, showing that their "Best Model" configurations (often with 24 or 28 hash bits) achieve competitive scores, sometimes approaching the upper bound of observed mentions. For example, for Dutch, the baseline is 0.63, while their best model achieves 0.93.

Limitations and Future Work

  • Language-Specific Phenomena: The authors acknowledge that language-specific phenomena, like morphological differences, significantly impact optimal parameter settings and model performance.
  • Noise in Data: The inherent noise in Wikipedia data is a challenge, especially for smaller datasets, and requires careful handling.
  • Future Work: The authors suggest extending their approach to incorporate more background knowledge about entities, use richer feature representations, and explore weak supervision. They also aim to add support for tuning models to desired levels of granularity, distinguishing between administrative entities and cities, for example. They also mention plans to replace their BoW features with multilingual embeddings in future work.

Summary updated

BibTeX
@inproceedings{kruit2023minimalist,
  author = {Benno Kruit},
  title = {{Minimalist Entity Disambiguation for Mid-Resource Languages}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2023},
}
Figure