Summarizing spoken conversations, such as business meetings or customer service calls, presents a unique set of challenges compared to summarizing structured text like news articles. Spoken language is often fragmented, interrupted, and non-linear, with multiple topics discussed simultaneously. This paper introduces a novel approach to Abstractive Community Detection (ACD), a critical sub-task designed to group related utterances into "communities" that can be summarized by a single abstractive sentence.
The Challenge of Spoken Dialogue
Traditional extractive summarization methods, which select the most important sentences from a text, often fail in conversational settings. Individual utterances like "Yeah, that works" or "I agree" are meaningless without their surrounding context. The goal of ACD is to identify specific utterances that together support a summary point, even if they are separated by time or other topics.
As shown in the figure above, different participants may contribute to various summary points (e.g., decisions about casing material) throughout a meeting. ACD aims to link these disparate fragments into coherent clusters.
Methodology: Hierarchical Self-Attention and Energy-Based Learning
The authors propose a sophisticated neural utterance encoder trained within an Energy-Based Modeling (EBM) framework to address the contextual and messy nature of human speech.
1. Hierarchical Utterance Encoder
The core of the model is a hierarchical encoder that processes utterances not in isolation, but within their "neighborhood." It employs three distinct self-attention mechanisms:
- Word Encoder (WE): Combines word embeddings with discourse features such as the speaker's role and the type of speech act (e.g., question vs. statement).
- Content-Aware Self-Attention: Analyzes a window of surrounding context (approximately 11 utterances before and after) to identify complementary information that clarifies the current utterance's meaning.
- Time-Aware Self-Attention: Incorporates a trainable decay mechanism to weight context based on temporal proximity, recognizing that closer utterances are generally more relevant.
- Utterance-Level Self-Attention: Produces a final, compact vector representation (embedding) of the utterance.
2. Energy-Based Learning Framework
Rather than using a standard classifier, the model is trained using Siamese and Triplet architectures. The Triplet Network approach proved particularly effective. By training on a trio of utterances (Anchor, Positive, and Negative), the model learns to minimize the "energy" (distance) between related utterances while maximizing the distance to unrelated ones. This allows the model to organize the embedding space based on relative semantic relationships rather than absolute constraints.
3. Overlapping Clustering
Once utterances are projected into the embedding space, the authors apply Fuzzy c-Means (FCM) clustering. This choice is vital because it allows for overlapping communities, reflecting how a single utterance can contribute to multiple summary points.
Key Insights and Results
- Context is Crucial: The model's performance improved steadily as the context window increased, peaking at around 11 utterances. This confirms that meeting utterances are often "ill-formed" and require significant history and future context to resolve.
- Triplet > Siamese: The Triplet architecture provided more robustness and flexibility in organizing the embedding space compared to Siamese networks.
- Superior Performance: The proposed model significantly outperformed baselines, including Hierarchical Attention Networks (HAN) and traditional TF-IDF approaches, particularly on the Omega Index, which measures the quality of overlapping clusters.
Conclusion and Future Work
This research demonstrates that energy-based self-attentive learning can effectively capture the complex, overlapping structure of human dialogue. While the current model excels at grouping utterances, future work remains to bridge the "extraction gap"—integrating the initial selection of summary-worthy utterances—and improving robustness against noise from Automatic Speech Recognition (ASR) systems.
Summary updated