Report on the Multilingual Euphemism Detection Task

Patrick P. C. Lee, Anna Feldman2024 · DOI

Abstract

This paper presents the Multilingual Euphemism Detection Shared Task for the Fourth Workshop on Figurative Language Processing (FigLang 2024) held in conjunction with NAACL 2024. Participants were invited to attempt the euphemism detection task on four different languages (American English, global Spanish, Yorùbá, and Mandarin Chinese): given input text containing a potentially euphemistic term (PET), determine if its use is euphemistic or not. We present the expanded datasets used for the shared task, summarize each team's methods and findings, and analyze potential implications for future research.

Introduction

Euphemisms serve as linguistic "cushions," softening the impact of sensitive or harsh realities—for instance, using "between jobs" instead of "unemployed." For Natural Language Processing (NLP), detecting euphemisms is a complex task of contextual disambiguation. A phrase may be euphemistic in one context but literal in another. Furthermore, euphemisms are deeply rooted in specific cultural and linguistic contexts. While previous research has focused primarily on English, the FigLang 2024 shared task expands this scope to include American English (EN), global Spanish (ES), Yorùbá (YO), and Mandarin Chinese (ZH).

Methods

Participating teams employed diverse strategies to tackle the detection task:

  1. Few-Shot Prompting (GPT-4): The top-performing team utilized GPT-4 with few-shot prompting. They observed that providing a formal definition of a euphemism alongside eight labeled examples (particularly those sharing the same PET) significantly enhanced the model's performance.
  2. Behavior-Guided Ensembles: Another approach involved an ensemble of XLM-RoBERTa models fine-tuned on "behavior-related" tasks such as sarcasm and sentiment detection. This method posits that euphemism usage is pragmatically linked to other social linguistic behaviors.
  3. Efficient Fine-Tuning (LoRA): To address computational efficiency, one team used Low-Rank Adaptation (LoRA) to fine-tune a multilingual DistilBERT model, aiming for performance comparable to larger models with a smaller footprint.

Datasets and the "Unseen PET" Challenge

The shared task provided human-annotated datasets across the four target languages, sourced from online articles, social media, and transcribed texts. A critical variable in the dataset was the "PET overlap"—the frequency with which specific euphemistic terms appeared in both training and test sets.

Distribution of Potentially Euphemistic Terms (PETs) and overlap across languages.

As shown in the data, English (EN) featured a high overlap of 44 terms, whereas Spanish (ES) had zero overlap. This discrepancy created an "Unseen PET" wall; models struggled significantly with Spanish because they could not rely on memorized terms from the training phase, highlighting a major hurdle in generalizing the concept of a euphemism.

Key Results

Systems were evaluated using Macro-F1 scores to balance precision and recall. The results demonstrated that all participating systems significantly outperformed the majority-class baseline.

Final leaderboard results showing Macro-F1 scores for participating teams and the baseline.

The GPT-4 few-shot approach (amri228) achieved the highest average Macro-F1 score of 0.73. Performance varied by language, with English reaching up to 0.83 F1, while Spanish peaked at 0.67 F1, correlating with the overlap metrics.

Discussion and Insights

Beyond raw scores, the task revealed several key insights:

  • Multilingual Synergy: Models trained on a combination of all four languages generally outperformed those trained on a single language, suggesting cross-lingual patterns in figurative language.
  • Pragmatic Knowledge: The effectiveness of providing definitions in prompts suggests that high-level linguistic tasks benefit greatly from explicit pragmatic guidance.
  • Contextual Reliance: Experiments that removed the PET during training forced models to rely on surrounding context, revealing that models often "cheat" by memorizing specific keywords rather than understanding situational nuance.

Limitations and Future Work

Current models remain overly dependent on specific terms encountered during training. Future research should focus on improving model robustness to unseen euphemisms and conducting deeper error analyses to understand why certain terms (e.g., "disabled") remain difficult to classify. Additionally, investigating how models encode politeness could further advance figurative language processing.

Summary updated

Abstract

This paper presents the Multilingual Euphemism Detection Shared Task for the Fourth Workshop on Figurative Language Processing (FigLang 2024) held in conjunction with NAACL 2024.Participants were invited to attempt the euphemism detection task on four different languages (American English, global Spanish, Yorb, and Mandarin Chinese):given input text containing a potentially euphemistic term (PET), determine if its use is euphemistic or not.We present the expanded datasets used for the shared task, summarize each team's methods and findings, and analyze potential implications for future research.
BibTeX
@misc{lee2024report,
  author = {Patrick P. C. Lee and Anna Feldman},
  title = {{Report on the Multilingual Euphemism Detection Task}},
  year = {2024},
  doi = {10.18653/v1/2024.figlang-1.15},
}
Figure