Abstract This paper presents a robust approach to spoken document clustering by utilizing Word Confusion Networks (WCNs) instead of standard 1-best transcripts. By incorporating multiple recognition hypotheses weighted by posterior probabilities, the proposed method improves clustering accuracy, particularly in data-sparse environments with high word error rates. Experimental results on the Switchboard corpus demonstrate that a posterior-weighted TF-IDF approach significantly enhances clustering performance for short audio segments where keyword redundancy is low.
Introduction: The Problem of ASR Noise
Automatic Speech Recognition (ASR) systems are rarely perfect, with Word Error Rates (WER) often ranging from 10% to 60%. In the context of document clustering—where files are grouped by topic—relying on the "1-best" transcript (the single most likely sequence of words) can be problematic. If the ASR system fails to correctly identify a single topical keyword, a document may be misclassified.
While one could theoretically use a full "word lattice" to capture all possible sequences, these structures are often too massive and noisy for unsupervised clustering algorithms to handle effectively. This paper addresses the gap between overly simplistic transcripts and overly complex lattices by employing Word Confusion Networks.
From Lattices to Word Confusion Networks
A Word Confusion Network provides a manageable data structure that maintains alternative word candidates while simplifying the graph topology of a lattice. As shown in the comparison below, a WCN organizes the audio into a linear sequence of time slots, where each slot contains several word hypotheses, each assigned a posterior probability (confidence score).
Methodology: Posterior Weighted TF-IDF
The core innovation of this research is the adaptation of the standard Term Frequency-Inverse Document Frequency (TF-IDF) metric to account for ASR uncertainty. The authors propose a three-step pipeline:
- Transcription into WCNs: The ASR system generates a WCN rather than a single text string.
- Posterior Weighted TF-IDF: Instead of simply counting word occurrences, the "Term Frequency" is calculated by summing the posterior probabilities of a word across all slots in the document. This effectively down-weights words with low confidence, reducing the impact of recognition noise.
- K-Means Clustering: The resulting numerical vectors are processed using a standard K-Means algorithm to group documents into topical clusters.
Experimental Results and Insights
The study utilized approximately 1,100 conversations from the Switchboard corpus, labeled with 10 distinct topics. To simulate real-world conditions, the authors tested the system on both full 10-minute calls and short "initial 10%" segments at various simulated WER levels (30% to 65%).
The Redundancy Buffer
In long documents, 1-best transcripts perform surprisingly well even with high error rates. This is due to keyword redundancy; if the ASR misses one instance of a topic-specific word, it is likely to catch a subsequent repetition. However, this redundancy disappears in short audio clips.
WCN Performance in Data-Sparse Scenarios
The benefit of WCNs is most pronounced in short audio segments. In these cases, having "backup" word hypotheses in the WCN is crucial for identifying the topic. The authors reported up to a 4% absolute improvement (a 12% relative improvement) in Normalized Mutual Information (NMI) compared to 1-best transcripts at a 50% WER.
The Noise Trade-off
While WCNs provide more information, they also introduce noise. The authors found that weighting by posterior probability is essential. Without this weighting, the inclusion of incorrect hypotheses can actually degrade clustering performance, especially in long documents where the sheer volume of low-confidence words can overwhelm the signal.
Conclusion and Future Work
Word Confusion Networks offer a powerful alternative to standard transcripts for spoken document clustering, particularly when dealing with brief audio or high error rates. By weighting terms by their ASR confidence scores, the system achieves greater robustness against recognition errors. Future research will focus on feature selection to filter irrelevant "noise" words from WCNs and exploring alternative weighting schemes for the Inverse Document Frequency component.
Summary updated