Automatic Segmentation and Summarization of Meeting Speech

Gabriel Murray, Pei-Yun Hsueh, Simon Tucker, Jonathan Kilgour, Jean Carletta, Johanna D. Moore, Steve RenalsNAACL 2007

TL;DR

This paper introduces the AMI Meeting Facilitator, a system designed to automatically segment and summarize spoken meetings. By leveraging a rich set of features from audio, video, and transcript data, the system aims to provide users with a more intuitive way to navigate and understand recorded meetings, paving the way for improved information retrieval and extraction from this data.

What Problem Does This Paper Address?

Navigating and extracting information from recorded meetings is a significant challenge. Meetings, especially multi-party ones, generate a large volume of spoken content that is difficult to sift through manually. Traditional methods of reviewing recordings are time-consuming and inefficient. This paper addresses the need for automated tools that can break down meeting content into meaningful segments and then condense these segments into concise summaries, making the information more accessible and actionable.

Methods

The AMI Meeting Facilitator is built upon two core components: segmentation and summarization.

  • Segmentation: This component divides a meeting into locally coherent segments, identifying topic boundaries. It employs a Maximum Entropy (Max-Ent) model trained on features extracted from audio and video recordings. The features used are diverse and include:
  • Conversational Features: Overlapping speech, silence duration, speaker activity similarity, cue words, and predictions from LCSEG (lexical cohesion statistics).
  • Lexical Features: Presence or absence of uni-grams within a segment.
  • Prosodic Features: Dialogue-act (DA) rate-of-speech, pitch (FO) and energy levels, silence within DAs, and DA duration.
  • Motion Features: Average speaker movement over short time windows.
  • Contextual Features: Dialogue act types and speaker roles.
  • Summarization: This component selects the most important utterances to create a summary. It uses a Support Vector Machine (SVM) trained on features that include:
  • Prosodic Features: Similar to those used in segmentation (DA rate-of-speech, pitch, energy, silence, pauses, duration).
  • Speaker Features: Dominance in the meeting, measured by percentage of dialogue acts, words, and speaking time.
  • Structural Features: Position of a dialogue act within the meeting and within a speaker's turn.
  • Term Weighting Features: Including tf.idf (term frequency-inverse document frequency) and a feature that captures how word usage varies across participants.

A Compression component is also mentioned, which further refines the summary by removing less important words from selected utterances based on user-specified compression levels.

Datasets

  • Segmentation: Trained on a set of 50 meetings separate from the input meeting.
  • Summarization: Trained on a set of 98 "scenario meetings."

The paper does not explicitly state the size or origin of these datasets beyond the number of meetings.

Key Insights and Ideas

  • Multi-modal Feature Fusion: The system's strength lies in its ability to integrate a wide array of features from different modalities (audio, video, transcript) and linguistic levels (prosodic, lexical, structural, contextual). This holistic approach is crucial for accurately understanding the nuances of spoken conversations.
  • Hierarchical Information Extraction: By first segmenting meetings into topic-based chunks and then summarizing these chunks, the system provides a structured way to access information, which is more effective than a flat summary.
  • User-Centric Control: The inclusion of a compression component suggests a focus on user control, allowing individuals to tailor the level of detail in the summary to their specific needs.

Key Results

The paper describes the process of training and testing the models but does not present specific quantitative results (e.g., accuracy scores for segmentation or summarization, compression ratios achieved). It states that the segmentation component classifies utterances as topic boundaries or not, and the summarization component ranks dialogue acts by importance. The ultimate goal is to provide visual aids for meeting interpretation and support downstream information extraction.

Limitations and Future Work

  • Lack of Quantitative Evaluation: The paper focuses on describing the system's components and methodology rather than presenting empirical results. This makes it difficult to assess the system's performance against existing benchmarks or its practical effectiveness.
  • Dataset Specificity: The reliance on specific meeting corpora for training might limit the generalizability of the system to different types of meetings or domains.
  • Future Work: The paper implicitly suggests that further work would involve rigorous evaluation of the system's performance and potentially extending its capabilities to other aspects of meeting analysis.

Summary updated

BibTeX
@inproceedings{murray2007automatic,
  author = {Gabriel Murray and Pei-Yun Hsueh and Simon Tucker and Jonathan Kilgour and Jean Carletta and Johanna D. Moore and Steve Renals},
  title = {{Automatic Segmentation and Summarization of Meeting Speech}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2007},
}
Figure