Estimating Key Speaker in Meeting Speech Based on Multiple Features Optimization

Wei Li, Yanxiong Li, Qianhua HeInternational Journal of Signal Processing Image Processing and Pattern Recognition 2015 · DOI

Abstract

This paper proposes to estimate key speaker in meeting speech based on multiple features optimization. First, each feature is defined and their differences between key speaker and other speakers are analyzed. Then, a decision function of multiple feature weighting is generated for estimating key speaker in meeting speech, and the genetic algorithm is used to optimize these coefficients of feature weighting. The methods are evaluated on three different meeting speech datasets. Experimental results show that the proposed optimization method obtains average accuracy of 93.3% for estimating key speaker, and gains average accuracy improvement by 9.7% and 4.1% compared with the previous method and the feature weighting method without optimization, respectively.

Introduction

In the age of massive digital archives, identifying the "key speaker"—the person with the most authority or influence—in a meeting recording is a vital step for automated content analysis and semantic understanding. Whether it's a press conference, a diplomatic summit, or a corporate interview, the key speaker’s words are usually the most important to index and analyze.

Identifying the "dominant" person in a room is surprisingly difficult for an algorithm. Previous research often relied on a single metric, like total talk time. While this works for simple meetings, it fails in complex scenarios. For example, in a press conference, a translator might speak for more total seconds than the leader they are translating for. Similarly, a talkative host in a forum might have more "turns" than the special guest. Existing solutions utilizing video features (like eye contact) are computationally expensive and fail if the speaker moves. The authors sought a universal approach using only audio data.

Methodology

The researchers proposed a multi-feature optimization approach. Instead of relying on one "golden" metric, they extracted four audio features for every speaker:

  1. Average Speaking Rate: Words per second.
  2. Total Length of Speaking Time: Cumulative time spent talking.
  3. Maximum Length of Single Speaking Time: The longest uninterrupted "monologue."
  4. Total Times of Speaking: How many times the speaker took the floor.

After normalizing these features, they created a decision function—a weighted sum of all four metrics. A Genetic Algorithm (GA) was used to find the optimal weights for these features. The GA "evolves" the best weighting coefficients by testing combinations against a training set, allowing the system to automatically learn which features are most indicative of a key speaker in different contexts.

Datasets

The study utilized 54 hours of audio across 152 recordings, categorized into:

  • Press Conferences: Featuring Chinese Premiers answering reporters, involving translators and many short questions.
  • Speeches: Primarily public addresses by President Obama.
  • Forums and Interviews: Interactive sessions with 3–10 participants.

Feature Analysis and Key Insights

Statistical analysis reveals a distinct "fingerprint" for key speakers across the chosen metrics. Key speakers don't just talk more; they talk differently.

The statistical distribution of the four chosen features shows clear separation between the key speaker and others across speaking rate, total time, maximum single duration, and frequency.
  • The "Think Twice" Effect: Key speakers generally have a slower average speaking rate (Figure a). This suggests leaders speak extemporaneously and choose words carefully, whereas question-askers often read from prepared scripts.
  • Dominance through Duration: Key speakers consistently have longer total speaking times (Figure b) and significantly longer "maximum single speaking times" (Figure c), reflecting their right to hold the floor.
  • The Failure of Frequency: "Total Times of Speaking" (Figure d) was the weakest individual predictor. In many meetings, a host or translator might jump in more frequently than the main guest.

Results and Discussion

The optimized multi-feature approach proved highly effective, achieving an average accuracy of 93.3% across all meeting types. This represents a 9.7% improvement over the previous state-of-the-art method (relying solely on total speaking length) and a 4.1% improvement over a simple equal-weighting scheme. The GA-optimized weights provided consistent performance (92.9% to 94.1%) across all three dataset categories.

Limitations and Future Work

Despite high accuracy, miscalculations can occur:

  • The Translator Trap: A translator’s total speaking length and frequency can still occasionally exceed the leader's.
  • Fast-Talking Leaders: High speaking rates might lead to misclassification.
  • Host Dominance: In informal forums, an active host can be mistaken for the key speaker.

Future work could involve adding contextual features or a dynamic weighting system that adjusts based on the detected meeting type. This research provides a powerful, computationally efficient baseline for navigating modern meeting data.

Summary updated

Abstract

This paper proposes to estimate key speaker in meeting speech based on multiple features optimization. First, each feature is defined and their differences between key speaker and other speakers are analyzed. Then, a decision function of multiple feature weighting is generated for estimating key speaker in meeting speech, and the genetic algorithm is used to optimize these coefficients of feature weighting. The methods are evaluated on three different meeting speech datasets. Experimental results show that the proposed optimization method obtains average accuracy of 93.3% for estimating key speaker, and gains average accuracy improvement by 9.7% and 4.1% compared with the previous method and the feature weighting method without optimization, respectively.
BibTeX
@article{li2015estimating,
  author = {Wei Li and Yanxiong Li and Qianhua He},
  title = {{Estimating Key Speaker in Meeting Speech Based on Multiple Features Optimization}},
  journal = {International Journal of Signal Processing Image Processing and Pattern Recognition},
  year = {2015},
  doi = {10.14257/ijsip.2015.8.4.03},
}
Figure