A Long Short-Term Memory Framework for Predicting Humor in Dialogues

Dario Bertero, Pascale Fung

TL;DR

This paper introduces a novel framework using Long Short-Term Memory (LSTM) networks to predict humor in dialogues, specifically identifying punchlines. By modeling the sequential context of conversations, their approach significantly improves humor prediction accuracy and recall compared to traditional methods, paving the way for more empathetic human-robot interactions.

What Problem Does This Paper Address?

Detecting and understanding humor, irony, and sarcasm in text has been a growing area of research. However, much of this work has focused on short, isolated texts like tweets or generic jokes. This paper tackles a less explored but crucial area: predicting humor within the dynamic and context-dependent environment of conversational dialogues. Unlike a standalone joke, humor in dialogue often relies heavily on the preceding setup and the overall conversational flow. The authors aim to build systems that can understand this nuanced humor, which is essential for creating more natural and engaging human-robot interactions.

Methods

The core of the proposed framework is a combination of a Convolutional Neural Network (CNN) for encoding individual utterances and an LSTM for modeling the sequential context of the dialogue.

  • Utterance Encoding (CNN): Each utterance is first processed by a CNN. This involves:
  • Input Features: Word tokens (one-hot vectors), character trigrams (to capture word morphology), and Word2Vec embeddings (for semantic meaning).
  • Convolution and Pooling: A sliding window captures local context within the utterance, followed by max-pooling to extract salient features into a single vector representing the utterance.
  • Sequence Modeling (LSTM): The encoded utterance vectors are then fed sequentially into an LSTM. LSTMs are well-suited for this task as they can maintain memory over long sequences, effectively capturing the "setup" that leads to a "punchline."
  • High-Level Features: In addition to the LSTM output, the authors incorporate several "high-level" features derived from linguistic analysis. These include structural features (word/sentence length), part-of-speech proportions, antonym proportions, sentiment scores, speaker information, and speaking rate. These features are concatenated with the LSTM output before the final prediction.
  • Baseline: A Conditional Random Field (CRF) model is used as a baseline, employing similar n-gram and language features.

Datasets

The authors built their corpus from the popular TV sitcom "The Big Bang Theory," seasons 1 to 6. They downloaded subtitle files (annotated with timestamps) and scripts to segment episodes into scenes and identify speakers. Crucially, they extracted audio tracks to identify the timestamps of canned laughter. An utterance was labeled as a punchline if it was followed by laughter within 1 second, assuming that utterances not followed by laughter served as the setup.

Key Insights and Ideas

  • Context is King for Dialogue Humor: The paper strongly emphasizes that humor in dialogues is not inherent to a single utterance but emerges from the interplay between setup and punchline, making sequential modeling critical.
  • CNNs are Effective for Utterance Representation: The CNN's ability to capture local lexical and semantic patterns within an utterance proved more effective than a simple LSTM encoder for this task.
  • LSTMs Excel at Capturing Conversational Flow: The LSTM's strength lies in its ability to accumulate context over time, allowing it to distinguish between a humorous punchline and a non-humorous utterance that might share similar lexical features.
  • High-Level Features Add Value: The inclusion of linguistic and structural features further boosted performance, suggesting that a multi-faceted approach to understanding humor is beneficial.
  • Reducing False Positives: The LSTM-based approach, particularly with the CNN encoder, was effective in reducing the number of false positives, meaning it was better at identifying true punchlines without incorrectly labeling other utterances.

Key Results

  • The proposed LSTM framework with high-level features achieved an F-score of 62.9%, a significant improvement over the best CRF baseline's F-score of 58.1%.
  • The LSTM model also demonstrated a higher recall (59.4%) compared to the CRF baseline (47.8%), indicating it identified more actual punchlines.
  • A comparison between CNN and LSTM as the initial encoding stage showed that the CNN encoder outperformed the LSTM encoder, achieving a 10% higher recall and 6% higher F-score. This suggests that for this specific task and dataset size, the CNN's local feature extraction was more beneficial.

Limitations and Future Work

  • Canned Laughter as Ground Truth: The reliance on canned laughter as the sole indicator of humor is a limitation. The authors acknowledge that canned laughter is sometimes inserted by producers to solicit a response to weak jokes, meaning it's not always a perfect proxy for genuine audience amusement.
  • Future Work: The authors plan to:
  • Adapt their model to predict humor from human-robot interactions.
  • Explore using more data.
  • Investigate deeper or bi-directional LSTMs.
  • Experiment with other sentence input encoders.

Summary updated

BibTeX
@inproceedings{bertero2016long,
  author = {Dario Bertero and Pascale Fung},
  title = {{A Long Short-Term Memory Framework for Predicting Humor in Dialogues}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2016},
  doi = {10.18653/v1/n16-1016},
}
Figure