Fuzzy Matching of Semantic Class in Chinese Spoken Language Understanding

Yanling Li, Qingwei Zhao, Yonghong YanIEICE Trans. Information and Systems 2013 · OpenReview

Abstract

Spoken Language Understanding (SLU) often faces challenges due to Automatic Speech Recognition (ASR) errors or incomplete user queries. This paper proposes a robust two-stage method to obtain semantic concepts from noisy utterances. First, a Conditional Random Field (CRF) model segments and labels entity names at the character level. Second, fuzzy matching based on similarity functions maps these entities to a reference dictionary. Experiments show that Dice and Cosine similarity measures achieve over 93% F1-measure, significantly outperforming traditional string-matching methods like q-gram and improved edit-distance.

Introduction

The bridge between a user’s voice and a computer's database is Spoken Language Understanding (SLU). In real-world applications, this bridge is often compromised by three primary factors:

  1. ASR Errors: Misinterpretation of tones and phonemes in Chinese leads to incorrect character output.
  2. User Incompleteness: Users frequently omit parts of titles (e.g., saying "Pleasant Goat" instead of "Pleasant Goat and Big Big Wolf").
  3. Spoken Disfluencies: Hesitations and repetitions disrupt standard linguistic patterns.

Traditional systems relying on exact matching fail under these conditions. This research introduces a pipeline designed to "correct" these errors dynamically.

System Architecture

The proposed SLU system integrates into a standard speech processing pipeline. Following ASR, the SLU component performs semantic class labeling and semantic understanding to generate feedback, as illustrated in the overall architecture.

Overview of the SLU system architecture within the speech processing pipeline.

Methodology: A Two-Stage Pipeline

The core contribution is a two-stage process that balances precision and computational efficiency.

Stage 1: Named Entity Recognition (NER) via CRF

Instead of matching entire sentences against a dictionary, the system uses a CRF model to identify potential entities. By operating at the single Chinese character level, the model avoids the pitfalls of word segmentation, which often fails when ASR errors occur.

Stage 2: Fuzzy Matching

When an identified entity lacks an exact dictionary match, a fuzzy matching process is triggered. This stage compares the labeled entity against dictionary entries using various similarity measures. This process allows the system to resolve incomplete queries into full database entries.

Example of the two-stage fuzzy matching process resolving an incomplete movie title.

The study evaluated four similarity measures:

  • Cosine Similarity (TF-based): Treats strings as character vectors.
  • Dice and Jaccard Similarity: Set-based measures of character overlap.
  • Improved Edit Distance: A sequence-based measure optimized for substring detection.

Experimental Results

The system was tested on a multimedia domain dataset (TV, movies, apps) containing over 48,000 training sentences and a specialized test set of 552 "noisy" sentences.

  • Accuracy: Both Cosine (TF-based) and Dice similarity achieved an F1-measure exceeding 93%.
  • Performance Gains: The Cosine approach improved the F1-measure by 8.8% over q-gram and 9% over improved edit-distance.
  • Efficiency: The pipeline processes queries in approximately 0.4 seconds, making it suitable for real-time applications.

Key Insights

A major finding was that set-based measures (Dice and Cosine) outperformed sequence-based measures (Edit Distance). This suggests that in spoken queries, the presence of key characters is a more reliable indicator of intent than their exact sequence or the number of edits required to transform the string. Furthermore, the character-level CRF acts as an effective pre-filter, significantly reducing the search space for the fuzzy matching stage.

Conclusion and Future Work

The proposed two-stage CRF and fuzzy matching approach provides a robust solution for Chinese SLU. Future research will explore open-domain performance and the integration of phonetic (Pinyin) similarity to further mitigate ASR-induced character confusion.

Summary updated

Abstract

Semantic concept in an utterance is obtained by a fuzzy matching methods to solve problems such as words' variation induced by automatic speech recognition (ASR), or missing field of key information by users in the process of spoken language understanding (SLU). A two-stage method is proposed: first, we adopt conditional random field (CRF) for building probabilistic models to segment and label entity names from an input sentence. Second, fuzzy matching based on similarity function is conducted between the named entities labeled by a CRF model and the reference characters of a dictionary. The experiments compare the performances in terms of accuracy and processing speed. Dice similarity and cosine similarity based on TF score can achieve better accuracy performance among four similarity measures, which equal to and greater than 93% in F1-measure. Especially the latter one improved by 8.8% and 9% respectively compared to q-gram and improved edit-distance, which are two conventional methods for string fuzzy matching.
BibTeX
@article{li2013fuzzy,
  author = {Yanling Li and Qingwei Zhao and Yonghong Yan},
  title = {{Fuzzy Matching of Semantic Class in Chinese Spoken Language Understanding}},
  journal = {IEICE Trans. Information and Systems},
  year = {2013},
  url = {https://openreview.net/forum?id=edxx1ljL4R},
}
Figure