A Simultaneous Recognition Framework for the Spoken Language Understanding Module of Intelligent Personal Assistant Software on Smart Phones

Changsu Lee, Youngjoong Ko, Jungyun SeoACL 2015 · DOI

Introduction

Spoken Language Understanding (SLU) serves as the critical intelligence layer in modern personal assistants like Siri or S-Voice. Its primary function is to map acoustic signals to semantic representations, identifying the user's intent through four key components: Named Entities (NE), Speech-acts (SA), Targets (TG), and Operators (OP).

Traditionally, SLU modules utilize a pipeline framework, where these tasks are executed sequentially. However, this architecture suffers from two significant drawbacks. First, error propagation occurs when a mistake in an early stage (e.g., NE recognition) cascades through subsequent modules, often leading to incorrect final intent classification. Second, the computational overhead of running multiple independent models is prohibitive for mobile devices with limited battery life and processing power.

Methodology: Simultaneous Recognition

To address these limitations, this research proposes a Simultaneous Recognition Framework that consolidates all four SLU tasks into a single Conditional Random Field (CRF) model. The core innovation lies in bridging the granularity gap between word-level tasks (NE recognition) and utterance-level tasks (SA, TG, OP classification).

The researchers introduced a "New Tag Addition Method" to unify these disparate tasks. By prepending "pseudo-morpheme" tags representing the Target, Operator, and Speech-act to the beginning of each user utterance, the entire problem is transformed into a sequence labeling task. This allows the CRF to learn dependencies between the global intent and local entities simultaneously.

The new tag addition method prepends pseudo-morphemes to utterances to unify recognition tasks.

By predicting all labels in a single pass, the model leverages feature synergy. For instance, identifying the "Target" as "Weather" provides a strong prior for identifying "Location" entities within the same sentence, creating a mutually reinforcing prediction environment.

Experimental Setup

The framework was evaluated using the MADS (Multi-Applications Dialogues for Smart phones) dataset, comprising 1,925 manually annotated user utterances across six domains: weather, clock, alarm, schedule, exchange, and traffic. The dataset includes 8 Named Entities, 28 Targets, 5 Operators, and 6 Speech-act tags. Performance was measured using 5-fold cross-validation, focusing on "utterance-level accuracy"—where a prediction is only considered correct if all four components are identified accurately.

Results and Discussion

The simultaneous framework demonstrated superior performance compared to the traditional pipeline baseline in both accuracy and efficiency.

Accuracy Improvements

The proposed method achieved an overall utterance-level accuracy of 83.48%, representing a 4% improvement over the pipeline framework (79.51%). This gain is attributed to the elimination of error propagation and the exploitation of inter-task dependencies.

Comparison of utterance-level accuracy between the simultaneous recognition framework and the pipeline framework.

Furthermore, the framework proved more robust than more complex joint models. In a comparison with a "triangular-chain CRF," the proposed simultaneous framework maintained high accuracy (83.48%) while the triangular-chain model's performance dropped to a mere 16.4% when handling all four tasks, suggesting that simpler unified architectures can be more effective for high-dimensional SLU tasks.

Computational Efficiency

Efficiency is paramount for mobile applications. The simultaneous framework reduced processing time by approximately 25%, handling 388 test utterances in 15 seconds compared to 19 seconds for the pipeline method. Statistical significance was confirmed via paired t-tests and Wilcoxon signed-rank tests (p < 0.05).

Conclusion

This paper presents a streamlined, simultaneous approach to SLU that outperforms traditional pipeline methods. By unifying recognition tasks through a novel tagging method, the framework reduces error propagation and computational load, making it ideal for mobile intelligent assistants. Future research should explore the scalability of this method to larger, open-domain datasets and its adaptation to modern neural architectures like Transformers.

Summary updated

BibTeX
@inproceedings{lee2015simultaneous,
  author = {Changsu Lee and Youngjoong Ko and Jungyun Seo},
  title = {{A Simultaneous Recognition Framework for the Spoken Language Understanding Module of Intelligent Personal Assistant Software on Smart Phones}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2015},
  doi = {10.3115/v1/p15-2134},
}
Figure