RoBoCoP: A Comprehensive ROmance BOrrowing COgnate Package and Benchmark for Multilingual Cognate Identification

Liviu P. Dinu, Ana Sabina Uban, Alina Maria Cristea, Anca Dinu, Ioan-Bogdan Iordache, Simona Georgescu, Laurentiu ZoicasEMNLP 2023 · DOI

Introduction

In historical linguistics, identifying cognates—words in different languages that share a common etymological origin—is fundamental to understanding language evolution. For instance, the Spanish flor and Italian fiore both descend from the Latin flos. Despite their importance, existing databases for Romance languages are often fragmented, noisy, or difficult to access for large-scale computational research.

Researchers have introduced RoBoCoP (Romance Borrowing COgnate Package), the largest and most comprehensive database of Romance language cognates and borrowings to date. This work provides both a massive "gold standard" dataset and a rigorous benchmark for automatic cognate identification using modern machine learning techniques, demonstrating that models can identify these linguistic relatives with high accuracy even when faced with challenging "false friend" examples.

Methodology

The creation of RoBoCoP involved a multi-step pipeline focused on data integrity and challenging evaluation:

  1. Data Extraction and Lemmatization: The team parsed electronic dictionaries for Romanian, Italian, Spanish, Portuguese, and French. To ensure consistency, they employed the Classical Language Toolkit (CLTK) to lemmatize Latin etymons, mapping various grammatical forms (like rex vs. regem) to a single root.
  2. Negative Sampling: To move beyond simple spelling matching, the researchers implemented "hard" negative sampling. In addition to random word pairs, they used Levenshtein-based sampling to select words that look orthographically similar but are etymologically unrelated. This prevents models from simply relying on surface-level spelling similarities.
  3. Model Architectures: Three distinct architectures were evaluated:
  4. Ensemble Models: Combining traditional machine learning (SVMs, Naive Bayes, XGBoost) with n-gram features from word alignments.
  5. Siamese CNNs: Deep learning models designed to extract parallel features from word pairs.
  6. Transformers: State-of-the-art sequence models processing character-level information.
Example of word extraction and etymon lemmatization in the RoBoCoP pipeline.

The RoBoCoP Dataset

The primary contribution, the RoBoCoP dataset, encompasses 125,598 words and 90,853 cognate pairs across five major Romance languages: Romanian (Ro), Italian (It), Spanish (Es), Portuguese (Pt), and French (Fr). Unlike previous resources limited to small Swadesh lists, RoBoCoP is exhaustive, covering the full vocabulary provided by source dictionaries and providing a more realistic representation of lexical relationships.

Results and Analysis

The benchmark results reveal several key insights into the nature of Romance linguistics and the current state of NLP:

  • High Accuracy on Challenging Tasks: On the most difficult task—distinguishing cognates from "hard" negatives—the best models achieved an average accuracy of 94%. When tested against random baselines, accuracy reached near-perfect levels (98-99%), suggesting that basic cognate detection is effectively a solved problem for these languages.
  • The "Romanian Difficulty": Performance consistently dipped for Romanian cognate pairs across almost all tests. This reflects Romanian's geographic isolation and its unique phonetic evolution and Slavic influences, which make its Latin roots harder to identify compared to Western Romance languages.
  • Orthography vs. Phonetics: Models were trained on both Graphic (spelling) and Phonetic (IPA transcriptions) representations. Combining both almost always yielded superior results. While phonetics represent the current state of the language, orthography often acts as a "fossil record," preserving historical markers that modern pronunciation has smoothed away.
  • The Power of Ensembles: Interestingly, Ensemble models using handcrafted linguistic features often outperformed deep learning Transformers. This suggests that for specific, structured tasks like cognate identification, domain-specific feature engineering based on linguistic principles remains highly effective.
Test accuracy across various language pairs and model architectures, highlighting the performance of ensemble and deep learning models.

Conclusion and Future Directions

By providing RoBoCoP to the research community, the authors have laid the groundwork for more sophisticated tools in machine translation, historical linguistics, and language learning. Future work aims to address semantic shifts—where words share origins but have diverged in meaning—and expand the dataset to include Romance dialects and minority languages such as Catalan, Occitan, or Sardinian.

Summary updated

BibTeX
@inproceedings{dinu2023robocop,
  author = {Liviu P. Dinu and Ana Sabina Uban and Alina Maria Cristea and Anca Dinu and Ioan-Bogdan Iordache and Simona Georgescu and Laurentiu Zoicas},
  title = {{RoBoCoP: A Comprehensive ROmance BOrrowing COgnate Package and Benchmark for Multilingual Cognate Identification}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2023},
  doi = {10.18653/v1/2023.emnlp-main.473},
}
Figure