TL;DR
This paper presents a story segmentation system for broadcast news in English, Mandarin, and Arabic, achieving competitive results using a rule-induction algorithm with automatically extracted acoustic and lexical features. A key finding is that acoustic features effective for English and Mandarin are not discriminative for Arabic, suggesting language-specific approaches are necessary.
What Problem Does This Paper Address?
Broadcast news programs are typically a collection of unrelated stories, often separated by anchor introductions or commercials. Automatically identifying the boundaries between these stories is crucial for downstream tasks like searching and summarizing broadcast news content. While significant research has focused on this problem for English and Mandarin, Arabic broadcast news segmentation remains largely unstudied, and existing state-of-the-art systems for English and Mandarin still have relatively high error rates.
Methods
The authors developed a story segmentation system for the NIGHTINGALE system. Their approach is based on binary classification, where potential story boundaries (hypothesized sentence boundaries) are classified as either a story boundary or not. They use a rule-induction algorithm called JRip, trained on automatically extracted features. These features fall into three categories:
- Lexical Features: These include measures of lexical similarity between text segments (using TextTiling and LCseg algorithms), and the identification of "cue-words" that frequently appear around story boundaries. For English, stemming is applied to the text.
- Acoustic/Prosodic Features: These capture characteristics of the speech signal within a sentence, such as F0 (pitch) and intensity statistics (minimum, maximum, mean, standard deviation), and F0 slope. They also include features related to speaking rate and vowel/rhyme length, hypothesized to indicate phrase-final lengthening at the end of topics.
- Speaker-Dependent Features: These leverage speaker diarization information to identify speaker participation patterns. Features include whether a speaker is the first or last speaker in a segment, the time since their last utterance, and the percentage of sentences spoken by the current speaker.
The system is trained using show-specific models, meaning a separate classifier is built for each news show.
Datasets
The training and evaluation data comes from the TDT-4 corpus. This corpus includes:
- English Broadcast News: 312.5 hours from 450 shows.
- Arabic News: 88.5 hours from 109 shows.
- Mandarin Broadcast News: 134 hours from 205 shows.
All shows aired between October 1, 2000, and January 31, 2001. The paper notes that TDT-5, which contains only text, was not used by their system.
Key Insights and Ideas
- Language-Specific Acoustic Cues: The most striking insight is that acoustic features that are highly indicative of story boundaries in English and Mandarin are not discriminative for Arabic. This suggests that vocal cues for topic transitions differ significantly across languages, possibly due to linguistic style or the language itself.
- Importance of Lexical and Acoustic Synergy: For English and Mandarin, lexical and acoustic features perform similarly and combine to yield improved results, highlighting the benefit of multi-modal approaches.
- Speaker Identity as a Boundary Indicator: Speaker participation patterns, captured by speaker-dependent features, significantly improve performance, especially for Mandarin. This is attributed to the strong correlation between speaker transitions and story boundaries in some Mandarin newscasts.
- Precision Over Recall: The authors prioritize precision (minimizing false alarms) over recall (minimizing missed boundaries). They argue that inserting erroneous boundaries can lead to more severe downstream errors in tasks like summarization and anaphora resolution than missing a boundary.
Key Results
- The system achieves success rates competitive with state-of-the-art systems on English and Mandarin.
- For English, the best performing model (L+A+S features) achieved a Cseg score of 0.0670.
- For Mandarin, the best performing model (L+A+S features) achieved a Cseg score of 0.0679.
- For Arabic, the best performing model (L+A+S features) achieved a Cseg score of 0.0850.
- Show-specific modeling consistently improved results across all metrics, reducing errors by 10-30%.
- The acoustic feature set performed poorly on Arabic data, while lexical and speaker-dependent features were still useful.
Limitations and Future Work
- Hypothesized Sentence Boundaries: The system assumes story boundaries only occur at hypothesized sentence boundaries. This assumption is true for only 47% of story boundaries in English, limiting overall performance. The average reference story boundary is about 9.88 words from an automatically recognized sentence boundary. (For Mandarin and Arabic, this is better at 69% and 62% respectively).
- Arabic Acoustic Feature Ineffectiveness: The paper explicitly states that acoustic features discriminative for English and Mandarin are not for Arabic. Future research will focus on studying vocal cues for segmentation in Arabic.
- Part-of-Speech Tagging: Reliable part-of-speech tagging for Arabic and Mandarin has not yet been obtained, which could potentially enhance lexical features.
- Comparison with Prior Work: While results are presented, direct comparison with some prior work is difficult due to differences in datasets, evaluation metrics, and whether segmentation was word-level or sentence-level. The authors are unaware of any reported story segmentation results for Arabic broadcast news prior to their work.
Summary updated