Deep Cross-Lingual Coreference Resolution for Less-Resourced Languages: The Case of Basque

Gorka Urbizu, Ander Soraluze, Olatz ArregiNAACL 2019

TL;DR

This paper introduces the first neural coreference resolution system for Basque, a less-resourced language. They demonstrate that a cross-lingual approach, trained on a larger English corpus and then applied to Basque, yields slightly better results than a monolingual system trained on a small Basque dataset, even without using any Basque training data. This highlights the potential of cross-lingual transfer learning for low-resource NLP tasks.

What Problem Does This Paper Address?

Coreference resolution, the task of identifying all expressions in a text that refer to the same entity, is crucial for many Natural Language Processing (NLP) tasks like summarization and question answering. While neural networks have significantly advanced coreference resolution for well-resourced languages like English, applying these techniques to less-resourced languages, such as Basque, is challenging. This is primarily due to the scarcity of annotated data, which is essential for training complex neural models. Building a robust coreference resolution system for Basque is therefore a significant hurdle.

Methods

The authors developed a neural coreference resolution system based on the "mention-pair" model, similar to systems used for Polish. This approach treats coreference resolution as a binary classification problem for every possible pair of mentions in a text.

Key methodological aspects include:

  • Neural Architecture: A fully connected neural network with three hidden layers (500, 300, and 100 neurons) and ReLU activation functions, followed by a sigmoid output layer.
  • Feature Engineering: For each mention pair, they extract features such as the average embeddings of words within the mention and its sentence, distance between mentions (in words and number of mentions), string matching, lemma matching, and a language indicator. They utilize pre-trained 300-dimensional FastText embeddings, which are beneficial for morphologically rich languages like Basque due to their subword information.
  • Cross-Lingual Embeddings: To bridge the gap between English and Basque, they employ the VecMap tool for unsupervised cross-lingual mapping of word embeddings. This allows them to leverage English embeddings for Basque without needing a bilingual dictionary.
  • Training: The monolingual system is trained on the Basque corpus, while the cross-lingual system is trained exclusively on the English corpus.

Datasets

  • EPEC-KORREF: A Basque corpus for coreference resolution, containing approximately 45,000 words and 12,000 mentions. This corpus was split into training, development, and test sets.
  • OntoNotes: A large English corpus (over 1 million words) used for training the cross-lingual system. The authors specifically used the newswire (nw) and broadcast news (bn) subsets, totaling around 825,000 words and 100,000 mentions.

Key Insights and Ideas

  • Cross-lingual transfer learning is effective for low-resource languages: Even with a significantly smaller Basque corpus, a system trained on a larger English corpus (and using cross-lingual embeddings) can achieve comparable, and slightly better, results than a monolingual system trained on the limited Basque data.
  • The importance of mention detection: The significant performance gap between using automatic and gold mentions for the monolingual system (over 14 points in F1) underscores the critical role of accurate mention detection in the overall coreference resolution pipeline.
  • FastText embeddings are beneficial for morphologically rich languages: The use of FastText embeddings, which incorporate subword information, is highlighted as a good choice for morphologically rich languages like Basque.
  • Small corpus size is a major bottleneck: The paper explicitly states that the Basque corpus is too small for a monolingual neural approach, reinforcing the need for cross-lingual strategies.

Key Results

  • Monolingual Basque System: Achieved 39.12 F1 (CoNLL metric) with automatic mentions and 53.19 F1 with gold mentions.
  • Cross-lingual System (English-trained, Basque-tested): Achieved 40.93 F1 (CoNLL metric) with automatic mentions and 54.46 F1 with gold mentions.
  • Comparison: The cross-lingual system slightly outperformed the monolingual system, despite not being trained on any Basque data.
  • Comparison to Previous Systems: The neural cross-lingual system's results (40.93 F1 auto, 54.46 F1 gold) were lower than previous rule-based and ML-based systems for Basque (55.98 F1 auto, 76.51 F1 gold for rule-based; 54.21 F1 auto, 73.94 F1 gold for ML-based), suggesting room for improvement.
  • Error Analysis: The system showed difficulties in linking pronouns to their antecedents, a common issue for neural and ML approaches compared to rule-based systems.

Limitations and Future Work

  • Limitations:
    • The performance of the neural systems is still lower than previous rule-based and ML-based systems for Basque.
  • The Basque corpus is very small, limiting the potential of monolingual approaches.
  • The system struggles with pronominal coreference and dropping pronouns, which are common in Basque.
  • Future Work:
    • Conduct an in-depth error analysis to better understand system behavior.
  • Train the same model on English to isolate the impact of the neural architecture versus corpus size and cross-lingual approach.
  • Investigate simpler models for the monolingual system.
  • Apply the cross-lingual approach to state-of-the-art neural architectures (e.g., K. Lee et al., 2018) to potentially improve performance and close the gap with gold mentions.
  • Test the cross-lingual system with different language pairs to build a universal coreference resolution system.

Summary updated

BibTeX
@inproceedings{urbizu2019deep,
  author = {Gorka Urbizu and Ander Soraluze and Olatz Arregi},
  title = {{Deep Cross-Lingual Coreference Resolution for Less-Resourced Languages: The Case of Basque}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2019},
}
Figure