Abstract
Scaling Spoken Language Understanding (SLU) systems to multiple languages is hindered by the high cost of producing labeled data. While data augmentation techniques like machine translation and generative modeling can synthesize training data, they often introduce noise that degrades model performance. This paper proposes a novel denoising framework that utilizes multiple models trained on diverse noisy datasets to supervise each other. Through strategies such as instance relabeling, co-training, and instance re-weighting, the proposed method achieves a new state-of-the-art on major benchmarks, improving performance by over 4 percentage points.
Introduction
Spoken Language Understanding (SLU) is a core component of modern voice assistants, encompassing intent detection and slot tagging. Despite its success in English, extending these capabilities to other languages remains a challenge due to the lack of high-quality labeled datasets. Researchers typically rely on zero-resource cross-lingual transfer, where models are trained on English data and augmented with synthetic target-language data. However, synthetic data is inherently noisy—translations may be inaccurate, and generative models can hallucinate. This noise often leads to poor generalization in the target language.
Methodology
The proposed framework addresses the noise problem by treating it as a manageable factor through multi-perspective learning. The approach consists of two primary stages:
1. Data Augmentation
Two distinct noisy datasets are generated to provide diverse perspectives:
- Machine Translation ($D_{trans}$): English data is translated using machine translation services, with word alignments established via tools like Giza++.
- Generative Modeling ($D_{gen}$): A generative model (e.g., mBART) is fine-tuned to produce new utterances based on specific dialog acts, effectively "dreaming up" variations.
2. Denoising Training
Instead of a single model, an ensemble of $K$ models is trained. During a "Relabeling Stage," three key strategies are employed to filter noise:
- Instance Relabeling: The ensemble predicts pseudo-labels for noisy data. If there is strong consensus that differs from the synthetic label, the label is updated.
- Co-Training: Models act as teachers for one another. A model identifies high-confidence (low-loss) examples to serve as training data for its peers.
- Instance Re-weighting: The variance between model predictions is used to identify uncertain examples, which are then down-weighted during training.
Experimental Setup
The framework was evaluated on two multilingual SLU benchmarks: SNIPS (English, Spanish, Thai) and MTOP (English, Spanish, French, German, Hindi, Thai). The experiments followed a zero-shot setup, where models were trained only on English and synthetic data, without any human-labeled target-language examples.
Results and Discussion
The proposed method significantly outperformed existing baselines, setting a new state-of-the-art for zero-shot cross-lingual SLU. On the SNIPS dataset, the method improved the F1/Intent Accuracy by 3.05 percentage points. On the more complex MTOP dataset, it achieved a 4.24 percentage point improvement in Exact Match Accuracy.
Qualitative analysis confirmed that the denoising module effectively corrected intent misclassifications and refined slot boundaries that were previously corrupted by translation errors. Furthermore, the independence of noise sources (translation vs. generation) proved crucial, as it allowed the models to effectively cross-verify each other's predictions.
Conclusion and Future Work
This work demonstrates that "crowd-intelligence" from multiple models can effectively mitigate the negative impact of noisy synthetic data in cross-lingual SLU. While the approach is computationally more intensive than single-model training, the performance gains are substantial. Future research will explore the applicability of this denoising framework to other cross-lingual tasks such as Named Entity Recognition and Question Answering, as well as optimizing the manual tuning required for co-training filtering rates.
Summary updated