TL;DR
This paper explores using word associations, a richer form of semantic relatedness than typical word embeddings, for humor recognition. They find that word associations outperform standard methods and introduce a novel approach to leverage "humor anchors" (key phrases that enable humor), though this latter technique surprisingly degrades performance.
What Problem Does This Paper Address?
Humor recognition is a notoriously difficult task for computers. While neural networks have shown promise, their "black box" nature makes it hard to understand why they succeed or fail. Traditional statistical methods offer more interpretability but often rely on simplistic semantic representations and ignore the crucial structural elements of jokes, like the setup and punchline. This paper aims to bridge this gap by combining the interpretability of statistical methods with more nuanced semantic and structural features derived from word associations and "humor anchors."
Methods
The core of this work lies in exploring word association features for humor recognition. Unlike word embeddings (like Word2Vec) which capture semantic similarity, word associations capture semantic relatedness, which the authors argue is more relevant for humor. They use two main methods for extracting these association strengths:
- Graph-based: This method relies on pre-compiled associative thesauri (like EAT and USF).
- ML-based: This approach uses a machine learning model trained on various features (including Word2Vec similarity) to predict association strengths between arbitrary word pairs, addressing the coverage limitations of graph-based methods.
They also investigate "humor anchors" (HAs), identified as minimal word spans crucial for humor. The idea is to use these HAs to focus semantic feature extraction on more meaningful word pairs, reducing noise. They employ a method similar to Yang et al. (2015) for HA extraction, which involves training a separate humor model to identify candidate anchors.
Finally, they build a humor classifier using a Random Forest model, incorporating these semantic features alongside a perplexity feature from a language model as a baseline.
Datasets
The authors evaluate their methods on two established humor datasets:
- Pun of the Day (PotD): Collected from the "Pun of the Day" website, with positive examples and negative examples from news, forums, and proverbs.
- 16000 One-Liner (OL): Consists of positive examples from humor websites and negative examples from news headlines, the British National Corpus, and proverbs.
Key Insights and Ideas
- Word Associations Trump Similarity: The paper provides strong evidence that word association features are more effective for humor recognition than standard word embedding similarity measures. This suggests that the nuanced, often asymmetric relationships captured by word associations are better suited to the cognitive processes involved in understanding humor.
- Relatedness vs. Similarity: The distinction between relatedness and similarity is crucial. While "beer" and "glass" are similar, "beer" and "glass" are related. Humor often relies on these related-but-not-similar connections, which word associations capture better.
- Humor Anchors: A Double-Edged Sword: The concept of targeting semantic features using humor anchors is intuitively appealing for reducing noise. However, the authors found that this approach decreased performance, suggesting that current HA extraction methods might be too noisy or that the definition of HAs doesn't perfectly align with the needs of semantic feature extraction for humor.
- ML-based Association Prediction is Robust: The ML-based approach for predicting word association strengths, which incorporates Word2Vec similarity as an input, still outperforms pure Word2Vec similarity, indicating it learns more than just simple vector proximity.
Key Results
- Performance Gains: The proposed model, using word association features, achieves performance comparable to or better than the state-of-the-art baseline (Yang et al., 2015) on both datasets, despite using significantly fewer features (28 vs. 318).
- Word Association Superiority: Word association features consistently outperform Word2Vec similarity across various configurations. For instance, on the 16000 One-Liners dataset, "Word Associations (ML)" achieved an F1 score of 0.726, compared to 0.658 for "Word2Vec."
- Humor Anchor Detriment: Incorporating humor anchors ("All Features (ML) + HA") led to a performance drop on both datasets, with the F1 score decreasing from 0.722 to 0.700 on PotD and from 0.751 to 0.700 on 16000 One-Liners.
- Graph vs. ML: While ML-based associations were expected to overcome coverage issues of graph-based methods, graph-based associations performed comparably and even better on the 16000 One-Liners dataset, possibly due to the longer text length increasing the likelihood of finding valid word pairs.
Limitations and Future Work
- Humor Anchor Quality: The authors acknowledge that the quality of extracted humor anchors is a significant limitation, with some anchors being incomplete or nonsensical. The performance of the HA extractor is directly tied to the performance of the underlying humor model.
- HA Definition Mismatch: The HA extraction method might not perfectly align with the needs of semantic feature extraction, as HAs don't explicitly indicate whether a word pair represents a setup-to-punchline or vice-versa.
- Dataset Bias: The authors noted concerns about potential bias when training language models on news corpora, given that negative examples are often drawn from news.
- Future Directions:
- Explore word association features on other humor tasks like relative humor ranking or humor generation.
- Investigate different word association datasets and strength metrics.
- Develop more robust humor anchor extraction methods or alternative ways to leverage joke structure.
- Explore more advanced ML-based prediction models for association strengths.
Summary updated