Abstract
Automatic Speech Recognition (ASR) systems require robust confidence annotation to identify recognition errors. This paper introduces the Latent Topic Similarity (LTS) algorithm, which extracts high-level semantic features by comparing the topic distribution of individual words to the overall context of an utterance. Using Latent Dirichlet Allocation (LDA), the method identifies "anchor words" to establish a stable context topic. Experimental results demonstrate that fusing LTS with traditional acoustic and language model features significantly reduces the Annotation Error Rate (AER), achieving a 27.68% relative reduction over baseline systems.
Introduction
While modern ASR systems have made significant strides, they remain prone to errors that are often difficult to detect using only low-level decoding information like acoustic likelihood or language model scores. Human listeners resolve such ambiguities by leveraging semantic and syntactic context. This paper proposes a method to bridge this gap by incorporating semantic "topic" information into the confidence annotation process. The goal is to create a feature that is statistically independent of the initial decoding process, thereby providing complementary information for error detection.
Methodology
The proposed approach centers on the Latent Topic Similarity (LTS) algorithm, which operates on the principle that correctly recognized words should align with the global topic of the sentence.
Topic Modeling with LDA
The system utilizes Latent Dirichlet Allocation (LDA) to discover latent topics within a large text corpus. LDA is a generative probabilistic model where each document is viewed as a mixture of various topics, and each topic is characterized by a distribution over words.
Context Topic Distribution and Anchor Words
To mitigate the impact of ASR errors on topic estimation, the algorithm identifies "Anchor Words"—recognized words with high posterior probabilities and strong topic orientations. The average topic distribution of these anchors defines the "context" of the utterance.
Similarity Measurement and Feature Fusion
For each word, the similarity between its topic distribution and the context distribution is calculated using Symmetric Kullback-Leibler (K-L) divergence. This LTS feature is then combined with standard features—Posterior Probability (PP), Acoustic Likelihood, and Language Model scores—and processed by a Support Vector Machine (SVM) to classify words as correct or incorrect.
Experimental Results
The algorithm was evaluated using the 863 Speech Recognition Database and the People Daily Corpus for topic model training ($K=50$). The performance was measured by the Annotation Error Rate (AER).
| Feature Set | AER (%) |
|---|---|
| Baseline (Accept All) | 8.89 |
| Posterior Probability (PP) | 7.53 |
| PP + LTS | 7.14 |
| Full Combination (PP + LLD + Lg + LTS) | 6.48 |
The inclusion of LTS led to a clear improvement over decoding-based features alone. Optimization of the anchor word selection parameters (Aratio and PPThresh) was critical for achieving the lowest AER.
Conclusion
The Latent Topic Similarity algorithm effectively captures semantic consistency as a proxy for recognition correctness. By utilizing an anchor-word strategy, the method remains robust even in the presence of ASR errors. While LTS is most effective when fused with traditional features, its ability to provide independent semantic signals makes it a valuable addition to modern ASR confidence annotation pipelines. Future work will focus on improving performance for short utterances where topic signals are inherently sparse.
Summary updated