Social role discovery from spoken language using dynamic Bayesian networks

Sibel Yaman, Dilek Hakkani-Tür, Gökhan TürInterspeech 2010 · OpenReview

Introduction

Identifying the social roles of participants in a conversation—such as distinguishing a host from a guest—is a fundamental task in Spoken Language Understanding (SLU). In broadcast settings, these roles provide critical context for downstream applications like automated summarization and information extraction. This research, presented at Interspeech 2010, explores how social roles can be automatically discovered by analyzing speaking styles and turn-taking patterns rather than just lexical content.

The Challenge of Role Identification

Previous research in role identification generally followed two paths: global analysis and local analysis. Global analysis examines a speaker's behavior across an entire broadcast (e.g., total talk time), which often misses the nuances of specific interactions. Conversely, local analysis assigns roles on a turn-by-turn basis. While local analysis captures conversational dynamics, it often fails to maintain consistency, potentially assigning different roles to the same individual at different points in the show.

The authors propose a middle ground: a model that captures sequential conversational dynamics while enforcing the logical constraint that a participant's role typically remains constant throughout a broadcast.

Methodology: Dynamic Bayesian Networks

The researchers focused on Speaking Style Features to identify roles. These features include disfluencies (filled pauses, repetitions), person addresses/mentions, and conversational prefaces (e.g., "well," "as you say").

To model these features, the study evolved from a standard Hidden Markov Model (HMM) to a more sophisticated Dynamic Bayesian Network (DBN). The core innovation is the "same-speaker constraint." In this model, the role assigned to a speaker at a specific turn depends not only on the previous speaker's role but also on the role assigned to that specific speaker the last time they spoke.

The DBN model incorporating the "Most Recent Role for the Same Speaker" (M nodes) to enforce consistency.

This "memory" mechanism acts as a correction tool, allowing the model to recover from local ambiguities by leveraging the speaker's established history in the conversation.

Experimental Setup

The study utilized English broadcast conversation corpora from the DARPA GALE program, encompassing 36 shows and 4,349 turns. The dataset included various genres such as talk shows and round tables, with roles categorized as Host (H), Guest (G), Audience Participant (A), and Reporting Journalist (J). The distribution was heavily weighted toward Hosts and Guests, who accounted for 97% of all turns.

Results and Discussion

The implementation of the same-speaker constraint led to significant performance gains over the baseline HMM.

Comparison of turn-level and speaker-level accuracy across different modeling approaches.

As shown in the results, turn-level accuracy improved from 79.2% with the HMM to 89.5% with the DBN. Speaker-level accuracy, determined by a majority rule for role assignment, also saw a notable increase. The DBN was particularly effective at correcting misidentified "Guest" roles during interactive segments and improved the identification of rarer roles like Journalists and Audience participants.

Conclusion

This work demonstrates that social roles are deeply encoded in the structure and style of speech. By using DBNs to enforce role consistency across a conversation, the researchers achieved a high degree of accuracy in role discovery. Future work aims to incorporate Dialog Act tags and test the system's robustness using fully automated feature extraction from noisy, real-world audio data.

Summary updated

BibTeX
@inproceedings{yaman2010social,
  author = {Sibel Yaman and Dilek Hakkani-Tür and Gökhan Tür},
  title = {{Social role discovery from spoken language using dynamic Bayesian networks}},
  booktitle = {Interspeech},
  year = {2010},
  url = {https://openreview.net/forum?id=uY_QLHQLEk},
}
Figure