TL;DR
This paper introduces SeLeCT, a novel lexical chaining approach for segmenting broadcast news into individual stories, outperforming existing methods on spoken news transcripts. It highlights the significant impact of spoken language characteristics on segmentation accuracy and proposes modifications to improve performance.
What Problem Does This Paper Address?
Automatically identifying distinct news stories within continuous streams of broadcast news is a crucial task for many downstream applications like topic tracking and first story detection. Traditional approaches often rely on lexical cohesion, the semantic relatedness of words within a text. However, segmenting spoken news presents a unique challenge due to its distinct linguistic characteristics compared to written text, leading to lower accuracy for existing methods. This paper aims to develop and evaluate a more robust segmentation system, particularly for spoken news, by leveraging lexical chains and accounting for the differences between spoken and written language.
Methods
The core of the proposed system, SeLeCT, is a three-component pipeline:
- Tokeniser: Preprocesses text by part-of-speech tagging, morphological analysis (singularizing plural nouns), and extracting noun phrases. It also employs fuzzy string matching to link related phrases (e.g., "George_Bush" and "President_Bush").
- Lexical Chainer: Identifies semantic relationships between tokens using the WordNet thesaurus and statistical word associations. It then builds "lexical chains" – groups of semantically related words that represent the text's cohesive structure. This is a single-pass clustering algorithm where tokens are added to the most semantically related chain.
- Boundary Detector: Uses the lexical chains to identify story boundaries. It calculates "boundary strength" between adjacent textual units (paragraphs) based on the number of chain start and end points. A threshold, derived from the mean of non-zero cohesive scores, is used to classify boundaries. An error reduction filter is applied to smooth out local maxima and remove spurious boundaries.
The paper also explores modifications:
- Verb Nominalization: Adapting the system to consider verbs more effectively by nominalizing them, addressing the higher verb usage in spoken language.
- Referential and Conjunctive Information: Integrating evidence from pronouns, determiners, and conjunctions in the boundary detection's filtering step to refine boundary identification, especially in clusters of potential boundaries.
Datasets
The paper evaluates its system on two test collections:
- CNN: Consisting of spoken news transcripts.
- Reuters: Consisting of written newswire articles.
Both collections were created by reorganizing 1000 randomly selected news stories into 40 files, each containing 25 concatenated stories. The basic unit of text for segmentation was the paragraph.
Key Insights and Ideas
- Spoken vs. Written Language Impact: Spoken news transcripts are significantly harder to segment accurately than written newswire. This is attributed to differences in lexical density and grammatical structure, with spoken language being more process-oriented (verb-heavy) and less product-oriented (noun/adjective-heavy) than written text.
- Lexical Chains are Powerful: Lexical chaining, by capturing semantic relationships, proves to be a strong basis for text segmentation.
- Verb Importance in Spoken Text: The higher prevalence of verbs and "low information" verbs in spoken language, which are often ignored by traditional lexical cohesion methods, negatively impacts segmentation performance. Nominalizing these verbs can significantly improve accuracy.
- Conjunctions and References as Cohesion Clues: Explicit references and conjunctions, particularly prevalent in conversational spoken text, provide valuable cues for identifying cohesive links between textual units and can be used to refine boundary detection.
- Metric Sensitivity: The WindowDiff metric is noted to penalize systems more than the Pk metric, but the overall ranking of systems remains consistent.
Key Results
- SeLeCT Performance: On the CNN (spoken) dataset, SeLeCT achieved better segmentation accuracy than TextTiling and C99 using both Pk and WindowDiff metrics.
- C99 Dominance on Written Text: For the Reuters (written) dataset, the C99 algorithm significantly outperformed both SeLeCT and TextTiling.
- Improvements from Modifications:
- Limiting input to nouns, adjectives, and nominalized verbs/adjectives reduced segmentation errors for C99 and TextTiling on CNN.
- Verb nominalization and integrating referential/conjunctive information improved SeLeCT's performance on CNN (WindowDiff error decreased from 0.253 to 0.225) but did not show improvement on Reuters.
- The effectiveness of referential and conjunctive information was much higher for CNN (51.66% of possible boundaries) than for Reuters (22.04%).
Limitations and Future Work
- Spoken Language Challenges: The paper acknowledges that spoken language's prosodic and paralinguistic features, not present in transcripts, contribute to segmentation difficulties.
- Future Work: The authors plan to compare SeLeCT with two recently proposed systems, U00 and CWM, which have shown marginally better performance than C99 on a specific test corpus. They also aim to further investigate the impact of spoken vs. written text differences on segmentation.
Summary updated