Proceedings of the 4th Workshop on Figurative Language Processing (FigLang 2024)

Debanjan Ghosh, Smaranda Muresan, Anna Feldman, Tuhin Chakrabarty, Emmy LiuNAACL 2024

TL;DR

This document presents the proceedings of the 4th Workshop on Figurative Language Processing (FigLang 2024), highlighting advancements in understanding and generating figurative language like metaphors, idioms, and sarcasm. It showcases new research, two shared tasks on euphemism detection and multimodal figurative language, and a keynote on LLMs' capabilities in this domain.

What Problem Does This Paper Address?

Figurative language, which enriches human communication by expressing complex ideas and emotions through non-literal means (e.g., metaphors, idioms, puns, irony, sarcasm), poses a significant challenge for Natural Language Processing (NLP) systems. Its ubiquity across all forms of discourse, from poetic to scientific, makes its automatic identification and interpretation crucial for many NLP applications. While recent advancements in large language models (LLMs) have shown promise, the field is still actively researching how to effectively process and understand these nuanced linguistic phenomena. This workshop series aims to foster a community dedicated to advancing research in this area.

Methods

The workshop features a variety of research methodologies, including:

  • Regular Research Papers: These papers explore diverse aspects of figurative language processing, such as:
  • Disagreement analysis in sarcasm detection.
  • Multimodal generation (e.g., image generation for figurative language).
  • Cross-lingual metaphor detection.
  • Annotation guidelines for identifying metaphors.
  • Metaphor detection models (e.g., expectation-realization models).
  • Idiom detection.
  • Distribution analysis of linguistic phenomena like personification.
  • Challenges in rhetorical figure detection.
  • Shared Tasks: Two shared tasks were organized to benchmark computational approaches:
  • Multilingual Euphemism Detection: This task involved developing models to classify texts in various languages as euphemistic or not. It expanded upon previous English-only datasets to include American English, Spanish, Yorùbá, and Mandarin Chinese, facilitating cross-lingual learning.
  • Multimodal Figurative Language: This task challenged participants to build models that not only identify the type of figurative language but also provide natural language explanations for their decisions, utilizing the FLUTE dataset which covers idioms, sarcasm, metaphors, and similes.
  • Keynote Talk: A keynote address discussed the current state of LLMs in recognizing, interpreting, and generating figurative language, questioning whether LLMs have effectively "solved" this challenge.

Datasets

  • Multilingual Euphemism Detection Shared Task: Datasets included texts from diverse sources (online articles, webpages, transcribed texts, social media posts) in American English (EN), Spanish (ES), Yorùbá (YO), and Mandarin Chinese (ZH). Texts contained up to three sentences with a potentially euphemistic term (PET), annotated for euphemistic (1) or non-euphemistic (0) usage.
  • Multimodal Figurative Language Shared Task: The FLUTE dataset was used, which is based on four types of figurative language: idiom, sarcasm, metaphor, and simile.
  • General Research: Various datasets are implied for the research papers, including corpus studies and datasets for specific tasks like sarcasm detection, metaphor detection, and idiom detection.

Key Insights and Ideas

  • LLMs and Figurative Language: The keynote suggests that the extensive exposure of LLMs to figurative language in their training data might be equipping them with the ability to understand and use it, prompting a re-evaluation of whether LLMs have "solved" this complex linguistic area.
  • Multilingualism is Key: The expansion of the euphemism detection task to multiple languages highlights the growing importance of cross-lingual understanding and transfer learning for figurative language processing.
  • Multimodality Matters: The multimodal figurative language task underscores the need for models that can integrate visual and textual information to better understand figurative expressions.
  • Annotation Challenges: The workshop acknowledges the complexities and challenges involved in annotating figurative language, as seen in the paper on guidelines for intentional linguistic metaphor.
  • Beyond Literal Meaning: Research continues to explore how to move beyond literal interpretations to capture the intended meaning of figurative expressions, which is crucial for tasks like sentiment analysis, dialogue systems, and content moderation.

Key Results

  • Multilingual Euphemism Detection: Three participating teams submitted system descriptions and achieved scores significantly above baselines, though below their reported validation metrics. The outcomes demonstrated the effectiveness of current approaches and highlighted the need for further research, particularly with LLMs, ensemble techniques, and task-specific strategies.
  • Multimodal Figurative Language: Four system papers were submitted, all based on transformer architectures. Participants explored various approaches, including using elaboration of the situation, sequential training on NLI datasets, and multi-sequence-to-sequence tasks. Two participants achieved the highest accuracy (@60) scores of 63.33.
  • Regular Research Track: Nine out of twenty-two submitted papers were accepted, covering a broad spectrum of figurative language processing topics.

Limitations and Future Work

  • LLM Performance: While LLMs show promise, their ability to fully "solve" figurative language is still under investigation, with ongoing research into their understanding and generation capabilities.
  • Cross-Lingual Transfer: Further research is needed to explore the broader impact of euphemistic terms (PETs) on model behavior and their potential connections to other linguistic tasks across languages.
  • Multimodal Integration: Developing more robust multimodal models that can effectively integrate visual and textual cues for understanding figurative language remains an active area of research.
  • Annotation Consistency: Establishing consistent and reliable annotation guidelines for various types of figurative language is an ongoing challenge.
  • Task-Specific Strategies: The need for further research into task-specific strategies and the exploration of LLMs, ensemble techniques, and their broader impact on model behavior is emphasized.

Summary updated

BibTeX
@inproceedings{ghosh2024proceedings,
  author = {Debanjan Ghosh and Smaranda Muresan and Anna Feldman and Tuhin Chakrabarty and Emmy Liu},
  title = {{Proceedings of the 4th Workshop on Figurative Language Processing (FigLang 2024)}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2024},
}
Figure