TL;DR
This paper tackles the challenge of automatically detecting whether a multi-word expression (MWE) is used literally or idiomatically across multiple languages. The authors propose two novel methods that leverage word senses (via glosses) and cross-lingual translations, showing that these linguistic resources significantly improve idiomaticity detection, especially in low-resource settings.
What Problem Does This Paper Address?
Understanding and detecting idiomatic expressions is a long-standing challenge in Natural Language Processing (NLP). Idioms are phrases whose meaning cannot be deduced from the literal meanings of their constituent words (e.g., "kick the bucket" doesn't literally mean to strike a pail). This non-compositionality makes them difficult for machines to interpret, especially in multilingual contexts where the same idiom might not exist or might be expressed differently. The SemEval-2022 Task 2 specifically aimed to address this by classifying MWEs as literal or idiomatic across various languages, posing a significant challenge for NLP systems.
Methods
The University of Alberta team developed two primary methods, both building upon a multilingual BERT (mBERT) baseline:
- defBERT (Glosses): This method enhances the mBERT input by incorporating glosses (definitions) of all possible senses for each word within the MWE. The intuition is that if an MWE is idiomatic, at least one of its words will be used in a non-standard sense, which might be revealed by comparing the MWE's context with the standard definitions of its words.
- Translation-based Method (MT): This approach leverages the idea that literal expressions are typically translated word-for-word, while idioms often lose their meaning when translated literally. The method translates the MWE in context and then uses a multilingual wordnet to check if the translated words share a common concept (multi-synset) with their original counterparts. If they don't, it suggests the MWE is idiomatic.
They also explored combining these methods and incorporating a UNATT heuristic, which exploits the observation that some MWEs are inherently literal or idiomatic regardless of context, based on their presence in the training data.
Datasets
The paper uses data from the SemEval-2022 Task 2 for multilingual idiomaticity detection. This dataset includes MWEs, their languages, and surrounding sentences for context. The authors also utilize existing lexical resources:
- BabelNet: A large multilingual semantic network providing word senses and glosses.
- Open Multilingual WordNet (OMW): Another multilingual lexical database.
- TreeTagger: Used for lemmatization and part-of-speech tagging.
- Google Translate API: For translating MWEs.
- BabAlign & FastAlign: For word alignment.
- OpenSubtitles2016: A parallel corpus used to augment alignment data.
Key Insights and Ideas
- Glosses as a powerful signal: Adding word definitions (glosses) to the input of a neural model significantly improves its ability to detect idiomaticity, even when the glosses are in English, simplifying application to low-resource languages.
- Translation reveals idiomaticity: The core idea that literal expressions translate literally while idioms don't is a robust signal for idiomaticity detection, especially when annotated training data is scarce.
- Inherent idiomaticity: Some MWEs are consistently idiomatic or literal across contexts, a property that can be exploited by heuristics like UNATT.
- Proper nouns are literal: The observation that proper nouns are consistently classified as literal by their models is a useful insight, though it might also lead to over-simplification.
Key Results
- The defBERT-BN-en method achieved the best results in the zero-shot setting for English.
- The combination of defBERT with the UNATT heuristic yielded the best performance in the one-shot setting, demonstrating its effectiveness for low-resource languages like Galician.
- The translation-based method (MT) showed promise, particularly when combined with defBERT, highlighting the utility of translation for idiomaticity detection when direct annotations are unavailable.
- The authors' best overall result ranked third in the one-shot setting.
Limitations and Future Work
- Google Translate errors: The paper notes that Google Translate sometimes incorrectly translates idiomatic expressions literally, leading to errors.
- Word alignment issues: Complex alignments (e.g., many-to-one) can cause problems for the translation-based method.
- Calqued expressions: Modern idiomatic expressions are increasingly "calqued" (borrowed directly from other languages), which can make them appear literal even when they are not, posing a challenge for both gloss-based and translation-based methods.
- Future work: The authors suggest further investigation into the role of multilinguality in semantics and exploring how these findings can be applied to other semantic tasks. They also acknowledge the need to address errors in their MT implementation for future work.
Summary updated