TL;DR
Researchers have developed PSC-Joint, a novel approach to contextual Automatic Speech Recognition (ASR) that remains accurate even when the list of "hotwords" (biasing phrases) grows very large. By jointly modeling semantic correlations at the list, phrase, and token levels and using a "divide-and-conquer" purification strategy, the system achieves up to a 28% relative improvement in F1 scores over existing methods.
What Problem Does This Paper Address?
Contextual ASR allows speech recognizers to prioritize specific phrases, such as contact names ("Call Zhihao"), song titles, or niche technical jargon. These are provided to the model as a biasing list.
The industry standard for this is cross-attention, where the model "looks" at the biasing list to find matches. However, cross-attention has a major weakness: volume sensitivity. As the biasing list grows from a few dozen to thousands of phrases, the model gets "confused" by distractors—phrases that sound similar to the target. This leads to two issues:
- Under-biasing: Failing to recognize a phrase that is in the list.
- Over-biasing: Incorrectly "hallucinating" a biasing phrase when the speaker didn't say it.
Existing solutions often try to train models with more distractors, but this frequently degrades performance on smaller, simpler lists.
Methods
The authors propose Purified Semantic Correlation Joint Modeling (PSC-Joint). The core idea is that at any given moment in a recording, only a tiny fraction of the biasing list is actually relevant.
1. Multi-Level Semantic Correlation
Instead of a single attention pass, the model calculates correlations at three granularities:
- List-level: A binary check—is any phrase in this list relevant to the current audio segment?
- Phrase-level: Which specific phrases in the list are likely candidates?
- Token-level: Which specific characters or word pieces within those phrases match the ASR's internal vocabulary?
By computing the intersection of these three levels, the model "spotlights" the most relevant information and ignores the noise.
2. Group Competitive Purification (GCP)
Jointly modeling three levels of correlation is computationally expensive. To solve this, the authors introduced a "divide-and-conquer" mechanism. The biasing list is split into groups, and phrases within each group "compete" based on their correlation scores. Only the winners move on to the final joint modeling stage. This not only speeds up inference but also physically removes distractors before they can cause confusion.
3. Backbone Integration
The system is built on top of Paraformer, a high-performance non-autoregressive ASR model. It uses a collaborative decoding strategy where the standard ASR output and the "biased" output are interpolated based on the list-level confidence score.
Datasets
The models were evaluated on three major Chinese speech corpora:
- AISHELL-1: 178 hours of standard Mandarin.
- KeSpeech: 895 hours of Mandarin and eight sub-dialects (testing real-world generalization).
- AISHELL-NER: A named entity dataset used to generate realistic biasing lists (Person, Location, Organization) ranging from 51 to nearly 1,200 phrases.
Key Insights and Ideas
- The "Limited Relevance" Principle: Regardless of list length, only a few phrases are ever relevant to a specific audio frame. PSC-Joint exploits this by filtering early and often.
- Intersection is Key: Modeling at just the token level is noisy; modeling at just the phrase level is too coarse. The "Aha!" moment is that the consistency across list, phrase, and token levels is a much stronger signal for relevance than any single level alone.
- Purification as a Performance Booster: Usually, pruning a list is a trade-off for speed. Here, the authors found that purification actually improves accuracy (F1 score) because it removes the "confusion" caused by similar-sounding distractors in a massive list.
Key Results
- Accuracy Gains: PSC-Joint-P (the purified version) achieved relative F1 score improvements of 21.34% on AISHELL-1 and 28.46% on KeSpeech compared to baseline contextual models.
- Robustness to Scale: While baseline models saw their performance tank as the biasing list grew from 51 to 1,196 phrases, PSC-Joint remained remarkably stable.
- Efficiency: The Group Competitive Purification (GCP) achieved a 92.06% retention rate of target phrases while significantly reducing the computational load, allowing the model's inference speed to scale nearly linearly with list length.
Limitations and Future Work
- Training Assumptions: During the training of the semantic predictor, the authors assumed each sentence contains only one biasing phrase to simplify the classification task. Real-world speech often contains multiple entities.
- Language Scope: The study focused exclusively on Chinese Mandarin. While the architecture is language-agnostic, its effectiveness on morphologically rich languages (like Finnish) or alphabetic languages (like English) remains to be seen.
- Future Direction: The authors suggest exploring even larger biasing lists (tens of thousands) and further optimizing the "purification" rounds to handle extremely high-volume industrial applications.
Summary updated