Word-Error Correction of Continuous Speech Recognition Based on Normalized Relevance Distance

Yohei Fusayasu, Katsuyuki Tanaka, Tetsuya Takiguchi, Yasuo ArikiIJCAI 2015

Abstract

This paper introduces a novel two-step word-error correction framework for continuous speech recognition. By integrating Normalized Relevance Distance (NRD) to capture long-distance semantic dependencies and a dual-stage Conditional Random Field (CRF) model to handle null transitions in confusion networks, the proposed system significantly reduces Word Error Rate (WER). Evaluation on the Corpus of Spontaneous Japanese (CSJ) demonstrates a 15.48% absolute reduction in WER compared to raw ASR output, outperforming traditional local N-gram models.

Introduction

Automatic Speech Recognition (ASR) systems often produce errors that are acoustically plausible but semantically inconsistent. Traditional language models, such as N-grams, are limited by their local context window, failing to capture long-distance relationships between words. Furthermore, ASR outputs in the form of confusion networks often contain "null transitions" (gaps where no word is predicted) that disrupt local linguistic patterns. This paper addresses these challenges by proposing a semantic-aware, two-step correction method that bridges the gap between local syntax and global semantics.

Methodology

The proposed system architecture is divided into a learning phase for the detection models and a test phase for real-time error correction.

System architecture for learning and testing the error detection model, highlighting the two-step re-ranking process.

1. Normalized Relevance Distance (NRD)

The core innovation for capturing long-distance context is the Normalized Relevance Distance (NRD). Based on the theory of Normalized Web Distance, NRD uses a search-engine-like approach (utilizing the Lucene indexing tool) to calculate semantic relatedness. Unlike simple co-occurrence metrics, NRD incorporates relevance weights (similar to tf-idf), allowing it to estimate similarity even for words located far apart in an utterance. Content words—nouns, verbs, and adjectives—act as "semantic anchors" that help determine if a candidate word fits the overall topic of the sentence.

2. Two-Step CRF Correction

The correction process is split into two stages to handle the complexity of null transitions in confusion networks:

  • Step 1 (With Null Nodes): A CRF model is trained on features including N-grams, acoustic confidence scores, and NRD scores. This pass identifies which null transitions in the confusion network should be deleted.
  • Step 2 (Without Null Nodes): Once the first pass identifies the null transitions to be removed, the system "cleans" the sequence. It then runs a second CRF model trained on this cleaned sequence. This allows local N-gram features to work much more effectively because the gaps that previously blocked them are gone.
Example of the two-step error correction process, showing the transition from null-node inclusion to a cleaned sequence.

Experimental Setup and Datasets

The researchers evaluated their method using the Corpus of Spontaneous Japanese (CSJ). The ASR training set included 953 lectures, while error correction training utilized 450 lectures (approximately 508,299 words). Testing was performed on 100 lectures (approximately 29,162 words). A reference corpus of 2,672 lectures from the CSJ was indexed using Lucene to provide the relevance scores needed for the NRD calculations.

Key Results

The proposed NRD-based method showed substantial improvements over both the baseline and standard correction techniques:

  • Baseline WER: 43.52% (Raw ASR output).
  • N-gram Correction: 38.82% (Standard local approach).
  • Proposed NRD Method: 28.04%.

This represents a 15.48% absolute reduction in WER from the baseline and a 10.78% improvement over the standard N-gram correction method. The results also indicated that the system is most effective when the initial WER is below 40%, as the semantic context becomes more reliable when more "anchor" words are correctly recognized.

Conclusion and Future Work

The integration of long-distance semantic context via NRD and a two-step CRF approach provides a robust solution for ASR error correction. By treating the removal of null transitions as a distinct sub-task, the model enables local language features to perform at their peak. Future work will explore including a wider range of word classes, such as postpositional particles and auxiliary verbs, to further refine the context and improve performance in high-error scenarios.

Summary updated

BibTeX
@inproceedings{fusayasu2015word,
  author = {Yohei Fusayasu and Katsuyuki Tanaka and Tetsuya Takiguchi and Yasuo Ariki},
  title = {{Word-Error Correction of Continuous Speech Recognition Based on Normalized Relevance Distance}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2015},
}
Figure