A Dual-Attention Neural Network for Pun Location and Using Pun-Gloss Pairs for Interpretation

Shen Liu, Meirong Ma, Hao Yuan, Jianchao Zhu, Yuanbin Wu, Man Lan

Abstract

Pun location involves identifying the punning word in a short text, while pun interpretation aims to uncover the two distinct meanings of that word. Traditional methods often rely on isolated Word Sense Disambiguation (WSD) or phonetic information. This paper introduces the Dual-Attentive Neural Network (DANN), which integrates word senses, pronunciation, and context for simultaneous location of homographic and heterographic puns. Additionally, we frame pun interpretation as a classification task using pun-gloss pairs. Experimental results on benchmark datasets demonstrate that our approach achieves state-of-the-art performance.

Introduction

Puns are a sophisticated form of humor that rely on linguistic ambiguity. Computational pun processing generally involves two steps: Pun Location, identifying the "punning" word, and Pun Interpretation, identifying the two distinct meanings (senses) that make the pun work. Puns are classified as homographic (same spelling, different meaning) or heterographic (different spelling, similar sound).

Previous models often struggled because they either ignored pronunciation or relied on Word Sense Disambiguation (WSD) tools that pre-selected a few meanings, often missing the subtle second meaning required for a pun. This paper introduces a unified model that handles both types of puns without the bias of pre-selected word senses.

Methodology

The authors propose the Dual-Attentive Neural Network (DANN) for location and a Pun-Gloss Pair method for interpretation.

Pun Location (DANN)

DANN uses a sequence labeling approach built on three pillars:

  • Contextual Embedding: Uses BERT to understand the word's role in the specific sentence.
  • Sense-Aware Module: Pulls all definitions for a word from WordNet and uses an attention mechanism to "attend" to the most relevant meanings automatically.
  • Pronunciation-Aware Module: Breaks words down into phonemes and uses attention to find phonetic similarities, crucial for catching heterographic puns.
The architecture of the Dual-Attentive Neural Network (DANN) showing the sense-aware and pronunciation-aware modules.

Pun Interpretation

Rather than predicting a sense from a list, the authors reframe the problem as a sentence-pair matching task. They create "pun-gloss pairs" by concatenating the punning sentence with a specific dictionary definition (gloss). A BERT-based classifier then decides if the definition matches one of the intended meanings of the pun.

Evaluation

The models were evaluated on the SemEval-2017 Task 7 dataset, containing 1,607 homographic puns and 1,271 heterographic puns. The authors used 10-fold cross-validation to ensure robust results comparable to previous state-of-the-art models like PCPR and SAM.

Results and Discussion

The DANN model achieved new state-of-the-art (SOTA) results across both tasks.

Pun Location Performance

DANN demonstrated superior performance in identifying punning words:

  • Homographic: Achieved an F1-score of 0.904 (a 1.5% improvement over the previous best).
  • Heterographic: Achieved an F1-score of 0.928 (a 0.6% improvement).
Comparison of DANN performance against previous state-of-the-art models for pun location.

Pun Interpretation Performance

The "Ours" model reached an F1-score of 0.247, a significant 9.16% improvement over the previous best baseline (Duluth at 0.155).

Key Insights

  • Attention over Filtering: DANN bypasses error propagation from WSD tools by looking at all possible senses and letting the attention mechanism decide.
  • Phonetic Integration: By projecting pronunciation into the same embedding space as semantics, the model can "hear" phonetic relationships.
  • Classification for Interpretation: Treating interpretation as a binary matching task leverages the deep linguistic knowledge of pre-trained models like BERT.

Conclusion

While the model is highly effective, challenges remain for words with many meanings. Future work includes incorporating sentiment analysis to improve text generation and exploring these techniques in broader creative writing AI.

Summary updated

Abstract

Pun location is to identify the punning word (usually a word or a phrase that makes the text ambiguous) in a given short text, and pun interpretation is to find out two different meanings of the punning word. Most previous studies adopt limited word senses obtained by WSD(Word Sense Disambiguation) technique or pronunciation information in isolation to address pun location. For the task of pun interpretation, related work pays attention to various WSD algorithms. In this paper, a model called DANN (Dual-Attentive Neural Network) is proposed for pun location, effectively integrates word senses and pronunciation with context information to address two kinds of pun at the same time. Furthermore, we treat pun interpretation as a classification task and construct pungloss pairs as processing data to solve this task. Experiments on the two benchmark datasets show that our proposed methods achieve new state-of-the-art results. Our source code is available in the public code repository.
BibTeX
@inproceedings{liu2021dual,
  author = {Shen Liu and Meirong Ma and Hao Yuan and Jianchao Zhu and Yuanbin Wu and Man Lan},
  title = {{A Dual-Attention Neural Network for Pun Location and Using Pun-Gloss Pairs for Interpretation}},
  booktitle = {NLPCC},
  year = {2021},
  url = {https://openreview.net/forum?id=H58XIp8AAG9},
}
Figure