TL;DR
This paper explores how incorporating speech and discourse features, beyond just text, can significantly improve automatic speech summarization for meetings. They found that features like speaker activity, turn-taking, and discourse cues, when combined with text-based methods, outperform traditional approaches and even simple text-only summarization.
What Problem Does This Paper Address?
Summarizing spoken dialogue from meetings is a challenging task. Spoken language is often disfluent, fragmented, and subject to automatic speech recognition (ASR) errors. Traditional text summarization methods, while effective for written documents, struggle to capture the nuances of spoken communication. Previous research has explored prosodic features (like pitch and duration), but this paper investigates a broader set of speech-specific characteristics, including speaker activity, discourse cues (e.g., "decide," "discuss"), listener feedback (e.g., "right," "exactly"), and dialogue act length. The goal is to see if these features can lead to more informative and accurate summaries than text-only approaches.
Methods
The authors propose and compare three summarization approaches:
- Speech and Discourse Features: This method uses Latent Semantic Analysis (LSA) to represent utterances. Crucially, they augment the standard term frequency matrix with various speech and discourse features before applying LSA. These features include:
- Speaker Activity: Information about who spoke, turn-taking patterns, the number of active speakers around a dialogue act, and speaker interaction levels.
- Discourse Cues: Presence of words indicating decision-making or planned actions.
- Listener Feedback: Detection of positive feedback cues.
- Keyword Spotting: Identification of frequent, non-stop words.
- Structural Features: Dialogue acts in the middle to later parts of a meeting are scored higher.
- Dialogue Act Length: Longer utterances are hypothesized to be more informative.
- LSA Centroid: This is a text-based approach that uses LSA. They construct a "centroid" representing the document's core themes by averaging keyword vectors derived from a large corpus. Dialogue acts are then represented as averages of their constituent word vectors, and extraction prioritizes dialogue acts most similar to the centroid.
- Combined Approach: This method merges the rankings from the first two approaches, hypothesizing that their different strengths will complement each other.
Datasets
- ICSI Meetings Corpus: Used for development and testing. This corpus consists of 75 meetings, each approximately one hour long.
- Broadcast News Corpus: Used as a large corpus for deriving word vectors for the LSA Centroid method.
- AMI Corpus: Mentioned as a target for future work.
The paper also introduces a new evaluation scheme based on weighted precision, using multiple human annotations of meeting transcripts. This is motivated by findings that standard n-gram based metrics like ROUGE may not correlate well with human judgments in this domain.
Key Insights and Ideas
- Beyond Text is Crucial: The core idea is that spoken dialogue has unique characteristics that, when leveraged, can significantly improve summarization beyond what text alone can achieve.
- Speaker Activity Matters: Features related to speaker activity (who is talking, how much, and when interaction occurs) are surprisingly powerful predictors of important utterances.
- Dialogue Act Length is a Strong Indicator: The length of a dialogue act was found to be a very strong feature, mirroring findings in other speech summarization research.
- ROUGE Limitations: The study reinforces previous observations that standard metrics like ROUGE may not be reliable for evaluating meeting summarization, highlighting the need for more nuanced evaluation methods.
- ASR Robustness: Summarization systems, particularly those using speech and structural features, are remarkably robust to ASR errors, suggesting that imperfect transcription doesn't cripple their performance.
Key Results
- Speech Features Dominate: The "Speech and Discourse Features" approach achieved the highest weighted precision (0.52), significantly outperforming the centroid and random approaches.
- Combined Approach Offers No Significant Gain: While not worse, the combined approach did not improve upon the speech features approach, possibly due to the extreme brevity of the summaries and the potential for non-overlapping content.
- Centroid Best for ROUGE: Interestingly, the LSA Centroid method performed best according to ROUGE-2 scores, though the differences between systems were less pronounced than with weighted precision.
- ASR Impact Minimal: Weighted precision scores on ASR output were not significantly worse than on manual transcripts for any approach. The centroid approach even scored slightly higher on ASR.
- Feature Importance: For the speech features approach, dialogue act length was the most impactful feature, followed by keyword spotting and discourse cues. Speaker activity features had a less significant impact.
Limitations and Future Work
- Summary Length: The authors acknowledge that the very short summary length (350 words) might have limited the benefits of the combined approach, as two good summaries might extract different, non-overlapping content.
- Manual Dialogue Acts: The experiments used human-segmented and labeled dialogue acts. Future work will incorporate automatic dialogue act detection and labeling.
- Machine Learning Integration: The authors plan to incorporate these features into a machine-learning framework (specifically, Support Vector Models) for future work.
- Exploring Speaker Status: Investigating features that help determine speaker status (who is leading, who is being deferred to) is a key area for future exploration.
- AMI Corpus Application: Applying the developed methods to the AMI corpus, which has more structured meetings, is planned to assess feature transferability.
Summary updated