Abstract
Detecting social meaning in dialogue remains a significant challenge for natural language processing, particularly when models must generalize across different conversational domains. This paper presents a framework that leverages Large Language Models (LLMs) to generate textual rationales—explanations of a speaker's intentions, assumptions, and implicit information. By augmenting dialogue with these rationales, the authors demonstrate significant performance improvements in social meaning classification, especially in zero-shot and few-shot transfer scenarios. The results suggest that machine-generated reasoning acts as a semantic bridge, allowing models to focus on underlying social dynamics rather than domain-specific surface text.
Introduction
Detecting "social meaning"—the subtle cues humans use to "read between the lines"—is a complex task for artificial intelligence. While humans can easily distinguish between a speaker being genuinely distressed versus using self-pity as a negotiation tactic, standard NLP models often overfit to specific keywords found in their training data.
This creates a major hurdle for domain transfer. For example, a model trained to recognize "resistance strategies" in a charity donation dataset (e.g., "I don't have money right now") might fail when applied to a Craigslist price negotiation, even though the underlying social strategy is identical. This research aims to bridge this gap by making the "opaque surface form" of text more transparent through machine-generated reasoning, transforming raw dialogue into a more interpretable format for classification models.
Methodology: Multi-faceted Prompting
The core of the proposed approach is a framework called Multi-faceted Prompting. Rather than asking an LLM to classify an utterance directly, the authors prompt it to generate three distinct types of rationales that explain the context of the speech:
- Intention (INT): The deliberate goal the speaker seeks to achieve.
- Assumptions (ASM): The biases or background beliefs the speaker holds.
- Implicit Information (IMP): Cues that are inferred rather than overtly expressed.
Once generated, these rationales are concatenated with the original dialogue text using [SEP] tokens. This augmented input is then fed into smaller, more efficient classifier models such as BERT, GPT-2, or T5. The study evaluates this rationale-augmented approach against baselines that only process raw text, focusing on both in-domain (ID) and transfer (TF) settings.
Experimental Setup
The researchers evaluated the framework on two distinct social tasks across four well-known datasets:
- Emotion Recognition in Conversations (ERC): Utilizing the Friends dataset (TV show transcripts) and IEMOCAP (scripted and improvised dyadic interactions).
- Resisting Strategies (RES): Utilizing P4G (Persuasion for Good - charity donations) and CB (Craigslist Bargain - price negotiations).
These datasets were selected because they share identical labels (e.g., "Joy" or "Counter-argument") but represent vastly different linguistic contexts, providing a rigorous environment for testing domain generalization.
Key Results and Analysis
The authors conducted an extensive suite of 2,340 experiments to validate the framework. The high-level findings include:
- Performance Gains: Rationale-augmented models outperformed baselines in the vast majority of cases. In transfer settings (TF), using all three rationale types led to significantly higher performance in 73.8% of experiments.
- Zero-Shot Success: The framework proved particularly effective in zero-shot transfer for emotion detection, showing significant gains 83.3% of the time.
- The "Universal Language" of Reasoning: The study suggests that while surface words change between a TV show and a laboratory study, the underlying reasoning behind an emotion remains consistent. Rationales act as a semantic bridge that helps models ignore domain-specific noise.
- The Impact of Intention: Among the three rationale types, "Intention" (INT) was frequently the most impactful single feature. Understanding why a speaker said something proved more useful for classification than understanding their background assumptions.
- Low-Data Regimes: Performance gains were most pronounced in 5-shot or 10-shot learning scenarios. This suggests that rationales provide the "common sense" context that small datasets typically lack.
Discussion and Limitations
While the framework shows clear benefits, it is not without limitations. The current pipeline relies on proprietary models (such as GPT-3.5) to generate rationales, which introduces costs and potential reproducibility issues if APIs change. Furthermore, generating three detailed rationales for every turn in a long conversation adds significant computational overhead and latency.
Additionally, the framework showed asymmetric benefits; it was more effective for Emotion Recognition (ERC) than for Resisting Strategies (RES). The authors hypothesize that emotions are more universal across human experience, whereas negotiation strategies are more tightly coupled to the specific "rules" of a given domain.
Conclusion
This research demonstrates that machine-generated rationales are a powerful tool for enhancing social meaning detection. By making implicit social cues explicit, the framework allows smaller models to achieve high performance even in data-scarce or cross-domain environments. Future work will focus on "distilling" this capability into smaller, open-source models (like Flan-T5) to reduce costs and expanding the framework to other complex social tasks such as empathy detection and formal argumentation.
Summary updated