Automatic Summarization of Conversational Multi-Party Speech

Michel GalleyAAAI 2006

Introduction

Summarizing multi-party meetings presents a unique set of challenges that distinguish it from traditional text summarization. While news articles and documents generally follow a structured hierarchy with clear topical markers, spoken conversation is inherently messy, characterized by disfluencies, lack of formal punctuation, and high Word Error Rates (WER). This paper proposes a two-stage framework designed to bridge the gap between raw conversational transcripts and concise, readable summaries. The approach combines content selection—identifying what is important—with utterance revision—cleaning and rephrasing the selected content for clarity.

The Challenge of Conversational Speech

In the context of the ICSI Meeting Corpus studied here, the average Word Error Rate is approximately 34.8%. Beyond technical noise, the social nature of meetings introduces structural complexity. Traditional extractive summarization, which simply selects the most representative sentences, often fails because it ignores the conversational context. For instance, a participant’s agreement ("I think that works") is meaningless in a summary unless the original proposal it refers to is also included. To address this, the research emphasizes that the importance of an utterance is often defined by the social reactions it elicits from other participants.

Methodology: A Two-Stage Framework

1. Content Selection via Dynamic Bayesian Networks

The first stage involves identifying salient segments of the conversation. The process begins with topic segmentation, dividing the meeting into distinct topical blocks. Once segmented, each utterance is classified as either "Summary" or "Non-Summary."

To capture the dependencies between speakers, the framework utilizes Switching-Parent Dynamic Bayesian Networks (DBNs). Unlike standard models that treat sentences as independent or sequentially linked, the Switching-Parent DBN models the speaker-addressee relationship. It recognizes that the "parent" of a statement (the utterance it responds to) changes dynamically based on the flow of conversation. By modeling these long-distance dependencies, the system can better identify clusters of related, high-value information.

2. Utterance Revision

Once the salient utterances are selected, they must be transformed into a written-style summary. This stage moves beyond simple sentence compression (the deletion of words) toward a more abstractive approach. Using synchronous grammars—a formalism often found in machine translation—the system "translates" disfluent spoken sentences into concise, grammatical summaries.

This revision model incorporates multiple features to determine word importance, including:

  • Syntactic Rules: Ensuring the resulting sentence remains grammatically sound.
  • Language Models: Predicting the likelihood of word sequences.
  • Prosodic Features: Utilizing acoustic data, such as vocal stress, to identify which words carry the most information.

Key Insights and Results

The Social Context as a Feature

A primary insight of this work is that social interaction is a statistically significant indicator of importance. An utterance that triggers multiple responses or a lengthy discussion among different participants is significantly more likely to be summary-worthy. By treating the conversation as a social network of interactions rather than a linear stream of text, the DBN approach significantly outperformed standard first-order Markov models.

Performance and Human Agreement

The topic segmentation component achieved state-of-the-art results on the ICSI Meeting Corpus. Interestingly, the research found that while meetings are chaotic, human annotators show a high level of agreement regarding topic boundaries. This validates the feasibility of automated segmentation as a foundational step for summarization.

Limitations and Future Directions

While the content selection phase is highly advanced, the utterance revision component remains an area for further development. The current "decoding" process—the search for the optimal revised sentence—is computationally expensive and struggles with scalability for longer sentences. Additionally, the author notes that future work should focus on more robust probabilistic modeling of the grammar rules learned during the training phase. A comprehensive evaluation of the end-to-end system remains the ultimate goal as these individual components are further optimized.

Summary updated

BibTeX
@inproceedings{galley2006automatic,
  author = {Michel Galley},
  title = {{Automatic Summarization of Conversational Multi-Party Speech}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2006},
}
Figure