Abstract
Pun location involves identifying the punning word in a short text, while pun interpretation aims to uncover the two distinct meanings of that word. Traditional methods often rely on isolated Word Sense Disambiguation (WSD) or phonetic information. This paper introduces the Dual-Attentive Neural Network (DANN), which integrates word senses, pronunciation, and context for simultaneous location of homographic and heterographic puns. Additionally, we frame pun interpretation as a classification task using pun-gloss pairs. Experimental results on benchmark datasets demonstrate that our approach achieves state-of-the-art performance.
Introduction
Puns are a sophisticated form of humor that rely on linguistic ambiguity. Computational pun processing generally involves two steps: Pun Location, identifying the "punning" word, and Pun Interpretation, identifying the two distinct meanings (senses) that make the pun work. Puns are classified as homographic (same spelling, different meaning) or heterographic (different spelling, similar sound).
Previous models often struggled because they either ignored pronunciation or relied on Word Sense Disambiguation (WSD) tools that pre-selected a few meanings, often missing the subtle second meaning required for a pun. This paper introduces a unified model that handles both types of puns without the bias of pre-selected word senses.
Methodology
The authors propose the Dual-Attentive Neural Network (DANN) for location and a Pun-Gloss Pair method for interpretation.
Pun Location (DANN)
DANN uses a sequence labeling approach built on three pillars:
- Contextual Embedding: Uses BERT to understand the word's role in the specific sentence.
- Sense-Aware Module: Pulls all definitions for a word from WordNet and uses an attention mechanism to "attend" to the most relevant meanings automatically.
- Pronunciation-Aware Module: Breaks words down into phonemes and uses attention to find phonetic similarities, crucial for catching heterographic puns.
Pun Interpretation
Rather than predicting a sense from a list, the authors reframe the problem as a sentence-pair matching task. They create "pun-gloss pairs" by concatenating the punning sentence with a specific dictionary definition (gloss). A BERT-based classifier then decides if the definition matches one of the intended meanings of the pun.
Evaluation
The models were evaluated on the SemEval-2017 Task 7 dataset, containing 1,607 homographic puns and 1,271 heterographic puns. The authors used 10-fold cross-validation to ensure robust results comparable to previous state-of-the-art models like PCPR and SAM.
Results and Discussion
The DANN model achieved new state-of-the-art (SOTA) results across both tasks.
Pun Location Performance
DANN demonstrated superior performance in identifying punning words:
- Homographic: Achieved an F1-score of 0.904 (a 1.5% improvement over the previous best).
- Heterographic: Achieved an F1-score of 0.928 (a 0.6% improvement).
Pun Interpretation Performance
The "Ours" model reached an F1-score of 0.247, a significant 9.16% improvement over the previous best baseline (Duluth at 0.155).
Key Insights
- Attention over Filtering: DANN bypasses error propagation from WSD tools by looking at all possible senses and letting the attention mechanism decide.
- Phonetic Integration: By projecting pronunciation into the same embedding space as semantics, the model can "hear" phonetic relationships.
- Classification for Interpretation: Treating interpretation as a binary matching task leverages the deep linguistic knowledge of pre-trained models like BERT.
Conclusion
While the model is highly effective, challenges remain for words with many meanings. Future work includes incorporating sentiment analysis to improve text generation and exploring these techniques in broader creative writing AI.
Summary updated