Abstract
Natural language understanding (NLU) converts sentences into structured semantic forms. The paucity of annotated training samples is still a fundamental challenge of NLU. To solve this data sparsity problem, previous work based on semi-supervised learning mainly focuses on exploiting unlabeled sentences. In this work, we introduce a dual task of NLU, semantic-to-sentence generation (SSG), and propose a new framework for semi-supervised NLU with the corresponding dual model. The framework is composed of dual pseudo-labeling and dual learning method, which enables an NLU model to make full use of data (labeled and unlabeled) through a closed-loop of the primal and dual tasks. By incorporating the dual task, the framework can exploit pure semantic forms as well as unlabeled sentences, and further improve the NLU and SSG models iteratively in the closed-loop. The proposed approaches are evaluated on two public datasets (ATIS and SNIPS). Experiments in the semi-supervised setting show that our methods can outperform various baselines significantly, and extensive ablation studies are conducted to verify the effectiveness of our framework. Finally, our method can also achieve the state-of-the-art performance on the two datasets in the supervised setting.
1. Introduction
Natural Language Understanding (NLU) is a critical component of task-oriented dialogue systems, responsible for mapping user utterances into structured semantic representations, such as intents and slots. A persistent challenge in NLU is the high cost of data annotation, which limits the scalability of deep learning models to new domains. While semi-supervised learning (SSL) typically leverages unlabeled text, it often ignores the potential of "pure semantic forms"—structured intents and slots without corresponding text—which are relatively inexpensive to synthesize using domain knowledge.
This paper introduces a dual task to NLU: Semantic-to-Sentence Generation (SSG). By establishing a closed-loop relationship between NLU (sentence $\rightarrow$ semantics) and SSG (semantics $\rightarrow$ sentence), the authors propose a framework that utilizes both unlabeled sentences and synthesized semantic forms to improve model performance.
2. Methodology
The proposed framework bridges the gap between natural language and structured semantics through a dual model approach.
2.1 Semantic-to-Sentence Generation (SSG)
The dual SSG model generates natural language sentences from structured semantic inputs. It employs a hierarchical BLSTM encoder to process slot-value pairs and a Semantically Controlled LSTM (SC-LSTM) decoder. A copy mechanism is integrated to ensure that specific slot values, such as city names, are accurately preserved in the generated output.
2.2 Dual Learning and Pseudo-Labeling
To leverage unlabeled data effectively, the framework employs two main strategies:
- Dual Pseudo-Labeling: The NLU model predicts semantic labels for unlabeled text, while the SSG model generates sentences for synthesized semantic forms. Both models are iteratively fine-tuned on these pseudo-labeled pairs.
- Dual Learning: This approach uses a reinforcement learning setup where NLU and SSG act as two agents in a closed loop. The models are updated via policy gradients based on two types of rewards: reconstruction rewards (measuring how well the original input is recovered after a full cycle) and validity rewards (ensuring generated semantics are logically consistent, such as checking if a specific slot is valid for a given intent).
3. Experiments and Results
The framework was evaluated on the ATIS (Airline Travel Information Systems) and SNIPS datasets.
3.1 Semi-Supervised Efficiency
In settings with limited labeled data (ranging from 5% to 50%), the dual framework consistently outperformed standard SSL baselines like sentence auto-encoders and template synthesis. Notably, with only 10% of ATIS training data, the method achieved a 93.53% Slot F1 score. Ablation studies revealed that pseudo-samples generated by the SSG model were more critical to performance gains than those from the NLU model.
3.2 State-of-the-Art Performance
When trained on full datasets, the framework achieved state-of-the-art results. On ATIS, it reached 96.37% Slot F1 and 98.54% Intent Accuracy. On SNIPS, it achieved 96.11% Slot F1 and 98.86% Intent Accuracy. Further integration with pre-trained BERT embeddings pushed the SNIPS Intent Accuracy to 99.14%.
4. Conclusion
By introducing the dual task of SSG, this work provides a robust framework for semi-supervised NLU. The ability to exploit both unlabeled text and synthesized semantic forms effectively addresses the data sparsity problem. Future work will explore joint modeling of intents and slots and investigate domain adaptation to scale dialogue systems to more complex applications.
Summary updated