TL;DR
This paper introduces ParaNames, a massive, multilingual corpus of entity names (people, locations, organizations) derived from Wikidata, covering over 400 languages. They demonstrate its utility by training a multilingual model for canonical name translation, achieving promising results across a diverse set of languages.
What Problem Does This Paper Address?
Building robust natural language processing (NLP) systems for a wide range of languages, especially those with fewer resources, is a significant challenge. A key bottleneck is the lack of comprehensive, parallel data for tasks like named entity recognition, linking, and translation. While resources like Wikipedia and Wikidata contain vast amounts of multilingual information, extracting and structuring this data into usable parallel corpora for entity names is non-trivial. This paper addresses the need for a large-scale, multilingual resource specifically for entity names to facilitate research and development in cross-lingual NLP.
Methods
The authors leverage Wikidata, a structured knowledge base, to construct their ParaNames corpus. They extract entity records and their associated names across numerous languages. A crucial step involves handling script usage, where they filter names to ensure they conform to the expected scripts for each language, using Wikipedia as an authoritative source. To provide context for downstream tasks, they infer high-level entity types (Person, Location, Organization) from Wikidata's complex type hierarchy.
For their experimental demonstration, they train multilingual Transformer-based models for canonical name translation. They experiment with different "special tokens" prepended to the input to provide the model with additional information, such as entity type, language of the non-English label, and script.
Datasets
The primary dataset introduced is ParaNames, a multilingual parallel name resource derived from Wikidata. It comprises over 118 million names for 13.7 million entities, spanning over 400 languages.
For evaluation, they use a test set derived from ParaNames, split into train, dev, and test sets with an 80/10/10 ratio. They specifically focus on translating entity names to and from English for a selection of 19 languages: Arabic, Armenian, Georgian, Greek, Hebrew, Japanese, Kazakh, Korean, Latvian, Lithuanian, Persian (Farsi), Russian, Swedish, Tajik, Thai, Vietnamese, and Urdu.
Key Insights and Ideas
- Wikidata as a Goldmine for Multilingual Names: The paper highlights the immense potential of structured knowledge bases like Wikidata for creating large-scale, multilingual resources that were previously difficult to assemble.
- Script Consistency is Crucial: Manually curating allowed scripts for each language proved more effective than automated methods for ensuring data quality in multilingual name translation.
- Entity Type Information Aids Translation: The inclusion of entity type special tokens, alongside language and script tokens, generally improved translation performance, suggesting that contextual information is valuable for disambiguation.
- Language Similarity Matters: Performance in name translation is strongly correlated with linguistic similarity and script compatibility between the source and target languages.
Key Results
- ParaNames contains over 118 million names for 13.7 million entities across more than 400 languages.
- When translating to English, models performed best on languages using the Latin script (e.g., Swedish, Vietnamese, Latvian).
- Translating to English was consistently worst for Hebrew, likely due to its vowel-less nature.
- When translating from English, performance generally mirrored the "to English" direction, with Swedish and Latvian performing well.
- Hebrew translation from English showed improved performance compared to translating to English, potentially due to the model learning to infer vowels.
- Thai translation from English was significantly less accurate than translating to English.
- Adding special tokens (language, type, script) generally led to statistically significant improvements over a baseline model, though the differences were small.
Limitations and Future Work
- Data Quality Nuances: While efforts were made to ensure script consistency, the paper acknowledges that some data quality issues might persist.
- Manual Script List Creation: The manual construction of allowed scripts for each language, while effective, is a labor-intensive process that could be a bottleneck for scaling to even more languages.
- Focus on Specific Entity Types: The current corpus focuses on Person, Location, and Organization. Future work could expand to include other entity types.
- Further Exploration of Special Tokens: While the paper experimented with special tokens, there's room for further investigation into optimal combinations and their impact on performance across a wider range of languages and tasks.
- Broader Applications: The authors suggest ParaNames can enable research beyond name translation, including named entity recognition and linking, especially for low-resource languages.
Summary updated