TL;DR
This paper introduces a novel unsupervised approach to dialogue summarization that leverages word graphs and a unique point-of-view (POV) conversion module. By focusing on path reranking within word graphs and then abstracting the output, the method achieves strong results across various dialogue domains without requiring labeled training data.
What Problem Does This Paper Address?
Dialogue summarization presents a unique challenge compared to traditional text summarization: it requires converting first and second-person speech into third-person reported speech. This "point-of-view" shift, along with the diverse nature of dialogues (e.g., short conversations vs. meeting transcripts), makes it difficult to create robust summarization models. A major bottleneck is the scarcity of diverse, annotated dialogue-summary pairs, especially for low-resource languages. This paper tackles this challenge by proposing an unsupervised method that doesn't rely on large amounts of labeled data.
Methods
The core of the proposed method is built upon Multi-Sentence Compression Graphs (MSCGs). Here's a breakdown of their approach:
- Word Graph Construction: They construct a word graph where nodes represent words and edges represent co-occurrence statistics. Sentences are represented as paths from a
<bos>(beginning-of-sentence) node to an<eos>(end-of-sentence) node. Overlapping words create new, potentially shorter paths that capture more information. - Keyword Extraction: Instead of traditional centrality measures, they use graph degeneracy to identify keyword nodes. Words belonging to the k-core subgraph are considered keywords.
- Path Reranking: A novel scheme for path reranking is introduced, where paths are scored based on keyword coverage. This simple keyword counting approach is found to be effective.
- Topic Segmentation (Optional): For longer texts, an optional topic segmentation step is applied, allowing the summarization algorithm to be run on smaller, topic-specific segments. This is determined by analyzing the topic distance between consecutive sentences.
- POV Conversion: This is a key contribution. They introduce a rule-based module to convert the semi-extractive summaries (generated from word graph paths) into fully abstractive, third-person reported speech. This module handles pronoun changes, modal verb transformations, and question conversion.
Datasets
The authors evaluate their model on several benchmark datasets across different domains:
- Meetings: AMI, ICSI
- Day-to-day conversations: DialogSum, SAMSum
- Interviews: MediaSum
- Screenplay: SummScreen
- Debate: ADS
They note that for MediaSum, reference summaries exhibit a strong positional bias towards the beginning of transcripts, which can favor simpler extractive methods like LEAD-3.
Key Insights and Ideas
- Simplicity is Effective: The paper demonstrates that a simple keyword counting scheme for path reranking in word graphs can outperform more complex baselines.
- POV Conversion is Crucial for Abstractiveness: The dedicated POV conversion module significantly improves abstractive summarization scores, even when applied to already abstractive systems. This highlights the importance of explicitly handling the shift to reported speech.
- Unsupervised Approach is Viable: The success across multiple datasets without labeled training data underscores the potential of unsupervised methods for dialogue summarization, especially in data-scarce scenarios.
- Topic Segmentation Benefits R1/RL: While pre-segmented transcripts yield higher R2 scores, their proposed topic segmentation method leads to better R1 and RL scores, aligning with their emphasis on keyword extraction.
Key Results
The paper reports strong performance across various datasets. Notably:
- On meeting datasets (AMI and ICSI), their method, especially when augmented with POV conversion, outperforms previous state-of-the-art results.
- For day-to-day, interview, screenplay, and debate summarization, their model consistently outperforms the LEAD-3 baseline on R1 and RL metrics.
- The POV conversion module consistently improves scores on both their own system and existing baselines.
Limitations and Future Work
The authors acknowledge several limitations and suggest future research directions:
- Keyphrase/Bigram Preservation: The model's R2 scores are inconsistent across benchmarks, suggesting a potential weakness in preserving keyphrases or bigrams. They suggest incorporating insights from keyphrase-based summarizers.
- Grammatical Coherence and Fluency: The current system intentionally omits complex path filters that could ensure grammatical validity or factual correctness, leading to potential issues with coherence and fluency (as noted in Table 4).
- Deep Learning Integration: They propose augmenting their system with deep learning, particularly graph neural networks for MSCG representation and advanced deep learning approaches for POV conversion.
- Tense Conversion: Tense conversion is left as a future research direction.
- Rule-Based vs. Learned POV: While their rule-based POV conversion provides a strong baseline, they anticipate that learned approaches will offer further improvements.
Summary updated