Abstract
Language models excel at compositionality but often struggle with non-compositional phenomena like idioms. This work introduces MultiCoPIE, a corpus of potentially idiomatic expressions (PIEs) in Catalan, Italian, and Russian. By fine-tuning XLM-RoBERTa on English PIE data and evaluating cross-lingual transfer, we demonstrate that models can effectively generalize idiomaticity detection to unseen languages. Performance is notably higher for "shared idioms"—those with direct English counterparts—highlighting the role of cross-lingual lexical overlap in model generalization.
Introduction
In natural language processing, the principle of compositionality suggests that the meaning of a complex expression is determined by the meanings of its constituent parts. Idioms like "kick the bucket" or "break the ice" defy this principle. Many are Potentially Idiomatic Expressions (PIEs), which can be interpreted either figuratively or literally depending on the context. While English has extensive resources for PIE disambiguation, other languages lack such data. This paper introduces MultiCoPIE to bridge this gap and explores the efficacy of cross-lingual transfer from English to Catalan, Italian, and Russian.
The MultiCoPIE Corpus
The MultiCoPIE corpus was manually curated by extracting idiom instances from the OSCAR web corpus. It includes 123 Catalan, 111 Italian, and 145 Russian idioms, totaling nearly 8,000 annotated instances. Each entry provides linguistic features such as semantic compositionality, part-of-speech tags for the idiom head, and English equivalents.
Methodology
The researchers used XLM-RoBERTa, a transformer model pre-trained on over 100 languages. The model was fine-tuned on the English MAGPIE corpus and the SemEval-2022 Task 2a dataset. Evaluation focused on three settings: zero-shot (testing on unseen idioms), one-shot (seeing one example of an idiom), and random (standard mixed training). The study also investigated the impact of surrounding context (sentences before and after the target) on disambiguation performance.
Results and Discussion
The "Shared Idiom" Advantage
A significant driver of performance is cross-lingual lexical overlap. Idioms with direct counterparts in English (e.g., Italian rompere il ghiaccio vs. English break the ice) are classified with much higher accuracy. For "shared and seen" idioms, accuracy reached 0.95 across all languages. This suggests that the model leverages shared semantic embeddings across languages for these equivalents.
Contextual Impact
Surprisingly, providing additional surrounding context did not significantly improve performance. The "signal" for idiomaticity appears to be largely contained within the sentence containing the PIE itself. In some instances, extra context even acted as a distractor for the model, leading to slightly lower precision.
Cross-Lingual Transfer Performance
The English-tuned model demonstrated strong zero-shot transfer capabilities. For Catalan and Italian, the cross-lingual model achieved F1-scores of 0.94 and 0.92 respectively, outperforming models trained on limited native data. However, for Russian, a monolingual baseline remained more effective (0.91 F1 vs. 0.89 F1), likely due to the language's higher morphological complexity and its substantial representation in the original XLM-R pre-training.
Conclusion
The introduction of MultiCoPIE enables more robust research into multilingual idiomaticity. The results confirm that cross-lingual transfer is a viable strategy for PIE disambiguation, especially when idioms share lexical or semantic roots across languages. Future work will expand this research to non-Indo-European languages and address the natural data imbalance between figurative and literal usages.
Summary updated