A Just-In-Time Keyword Extraction from Meeting Transcripts

Hyun-Je Song, Junho Go, Seong-Bae Park, Se-Young ParkNAACL 2013

TL;DR

This paper introduces a novel "just-in-time" keyword extraction method for meeting transcripts that prioritizes recent utterances and topic relevance. By modeling temporal decay and filtering history based on current topic, it significantly improves keyword extraction accuracy compared to existing methods.

What Problem Does This Paper Address?

Extracting keywords from meeting transcripts is valuable for quickly grasping key discussion points. However, traditional keyword extraction methods, designed for static written documents, are ill-suited for the dynamic nature of meetings. In meetings, the relevance of information changes rapidly: recent utterances are generally more important than older ones, and only utterances related to the current topic are truly useful for understanding it. Existing methods often treat all historical utterances equally, leading to a dilution of relevant information and a drop in performance, especially when dealing with the continuous flow of spoken language. This paper aims to bridge this gap by developing a method that extracts keywords in real-time, considering both the temporal recency and topical relevance of preceding utterances.

Methods

The core of the proposed method is a graph-based approach that represents utterances and their relationships.

  • Graph Representation: Both the current utterance and the history of preceding utterances are modeled as graphs.
  • Current Utterance Graph (G1): Nodes are potential keywords (filtered nouns) from the current utterance, and edges represent co-occurrence within a defined window.
  • History Graph (G2): Nodes are keywords from previous utterances, and edges represent their co-occurrence. Edge weights in G2 are crucial and are updated using a "forgetting curve" model to reflect temporal importance.
  • Subgraph Extraction (G3): To address topic relevance, a subgraph of G2 is extracted that maximally covers the current utterance graph (G1). This ensures only relevant historical information is considered.
  • Expanded Graph (G4): G1 and G3 are merged to create an expanded graph that combines the current utterance's content with relevant historical context.
  • Keyword Extraction: The TextRank algorithm is applied to G4 to identify keywords, which are essentially the "hub nodes" of this combined graph.
  • History Update: The extracted keywords from the current utterance are then incorporated into the history graph (G2) for future utterances, with their weights updated according to the forgetting curve.

The novelty lies in the explicit modeling of temporal decay (recent utterances are more important) and topic relevance (only utterances related to the current topic are considered) within the graph framework.

Datasets

The paper evaluates its method on two datasets:

  • National Assembly Transcripts (Korean): This corpus consists of two meetings from the 305th assembly record. The first meeting (1,280 utterances) was used for keyword extraction, and the second meeting (573 utterances) was used for parameter tuning (window size, damping factor, threshold). Keywords were manually annotated by three annotators, with words agreed upon by at least two annotators being selected.
  • ICSI Meeting Corpus (English): This corpus, commonly used for meeting summarization and keyword extraction, consists of 26 meeting transcripts. For this study, 6 transcripts were used for development and the remaining for testing. Keywords were annotated at the topic segment level, requiring an adaptation of the proposed utterance-level extraction method.

Key Insights and Ideas

  • Temporal Decay is Crucial: The performance significantly drops when temporal history is not considered, highlighting that recent information is inherently more valuable in dynamic conversations.
  • Topic Relevance Matters: Filtering the history graph to include only relevant preceding utterances (those sharing words with the current utterance) is essential. Simply using the entire history graph leads to poorer results.
  • Graph-Based Approach is Effective: Representing utterances and their relationships as graphs allows for efficient integration of temporal and topical information.
  • Forgetting Curve for Temporal Modeling: The use of a forgetting curve effectively models the natural decline of memory retention over time, giving more weight to recent events.
  • "Just-In-Time" is Key: The ability to extract keywords as utterances occur, rather than waiting for the entire meeting, is a significant advantage for real-time applications.

Key Results

  • National Assembly Transcripts: The proposed method achieved an F-measure of 0.533 and a Weighted Relative Score (WRS) of 0.421. This significantly outperformed baseline methods like TextRank (F-measure: 0.478, WRS: 0.387) and TFIDF (F-measure: 0.481, WRS: 0.394).
  • ICSI Meeting Corpus: The proposed method achieved an F-score of 0.334 and a WRS of 0.533, outperforming unsupervised methods (TFIDF-Liu, TextRank-Liu) and even a supervised method (ME model).
  • Impact of Temporal History: When temporal history was ignored (all utterances treated equally), the F-measure dropped from 0.533 to 0.518 on the Korean dataset.
  • Impact of Topic Relevance: A variant that considered temporal history but not topic relevance showed improved performance over baselines but still underperformed the proposed method, demonstrating the importance of topic filtering.

Limitations and Future Work

  • Handling Irrelevant Utterances: The paper acknowledges that the method might extract keywords even for irrelevant or very short utterances. However, it suggests that such utterances would likely have low scores and thus be filtered out by the threshold.
  • Parameter Sensitivity: While parameters like window size (W), damping factor (d), and threshold (θ) were tuned, their optimal values might vary across different types of meetings.
  • Future Work: The paper implicitly suggests exploring more sophisticated ways to model topic relevance and potentially extending the method to other spoken language scenarios beyond meetings. The evaluation on the ICSI corpus also hints at the need for robust methods to adapt utterance-level extraction to segment-level tasks.

Summary updated

BibTeX
@inproceedings{song2013just,
  author = {Hyun-Je Song and Junho Go and Seong-Bae Park and Se-Young Park},
  title = {{A Just-In-Time Keyword Extraction from Meeting Transcripts}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2013},
}
Figure