Self-training with Two-phase Self-augmentation for Few-shot Dialogue Generation

Wanyu Du, Hanjie Chen, Yangfeng Ji

Abstract

In task-oriented dialogue systems, generating natural language responses from structured meaning representations (MRs) is a critical task. However, the high cost of manual annotation often limits the availability of training data. While self-training—leveraging models to generate their own training data—is a common solution, it frequently introduces noise or uninformative examples. We propose SA$^2$, a two-phase self-augmentation framework. The first phase selects informative MRs based on predictive uncertainty, while the second phase refines response generation through latent representation aggregation. Our experiments on FewShotWOZ and FewShotSGD benchmarks demonstrate that SA$^2$ significantly outperforms existing self-training methods.

Introduction

Natural Language Generation (NLG) in task-oriented systems involves mapping Meaning Representations (MRs), such as inform(food=seafood, area=centre), into natural sentences. In "few-shot" scenarios where only a handful of examples are available, models struggle to capture domain-specific nuances.

Traditional self-training attempts to bridge this gap by using a fine-tuned model to label unlabeled data. However, this approach faces two primary hurdles:

  1. Noisy Data: Models may generate hallucinated slots or grammatically incorrect sentences.
  2. Uninformative Data: Models often select examples they are already confident in, which provides little new signal for learning.

The SA$^2$ Framework

To address these challenges, we introduce the Two-phase Self-augmentation (SA$^2$) framework. This method ensures the model learns from data that is both relevant to the domain and challenging enough to provide new information.

Overview of the SA$^2$ framework, illustrating the two-phase process from unlabeled data to fine-tuning.

Phase I: Uncertainty-based Data Selection

The first phase focuses on identifying the most "informative" MRs from the unlabeled pool. We utilize Monte Carlo (MC) Dropout to estimate the model's uncertainty. By performing multiple forward passes with different dropout masks, we calculate two key metrics:

  • Predictive Mean ($\mathbb{E}[p_\theta]$): This measures how "familiar" the model is with the input. A high mean suggests the data is in-domain and relevant.
  • Predictive Variance ($Var[p_\theta]$): This measures the model's sensitivity or "uncertainty." High variance indicates the model has not yet mastered the specific patterns in that input.

The core innovation lies in selecting data from the "Goldilocks" zone: high mean and high variance. This targets examples that are relevant but difficult for the current model.

The uncertainty quadrant used for data selection, highlighting why high mean and high variance examples are preferred.

Phase II: Response Refinement

After selecting the optimal MRs, the system must generate high-quality target text. To mitigate the risk of the model overfitting to its own errors or hallucinations, SA$^2$ employs a refinement strategy. Instead of a single decoding pass, the model aggregates multiple perturbed latent representations. By averaging the hidden states from several passes (using dropout), the model produces a "smoothed" and more accurate response, which serves as the pseudo-label for the final fine-tuning stage.

Experimental Results

We evaluated SA$^2$ on two major benchmarks: FewShotWOZ (7 domains, ~50 examples/domain) and FewShotSGD (16 domains, ~35 examples/domain).

Performance Metrics

The proposed method consistently outperformed state-of-the-art baselines like SC-GPT and AUG-NLG:

  • BLEU Scores: On the FewShotWOZ Restaurant domain, SA$^2$ achieved a BLEU score of 36.48, a significant improvement over the 34.62 achieved by SC-GPT.
  • Slot Error Rate (ERR): SA$^2$ proved highly effective at reducing hallucinations. In the "Train" domain of FewShotWOZ, the error rate dropped from 1.90 to 1.62.
  • Human Evaluation: In blind tests, human judges consistently rated SA$^2$ higher in both informativeness and naturalness compared to standard self-training variants.

Conclusion

The SA$^2$ framework demonstrates that the quality of synthetic data is more important than the quantity. By combining uncertainty-based selection with latent representation refinement, we can effectively train high-performing dialogue models even with extremely limited human-annotated data. While the computational cost of MC Dropout is a consideration, the gains in accuracy and reliability make it a compelling approach for few-shot NLG tasks.

Summary updated

Abstract

In task-oriented dialogue systems, response generation from meaning representations (MRs) often suffers from limited training examples, due to the high cost of annotating MR-to-Text pairs. Previous works on self-training leverage fine-tuned conversational models to automatically generate pseudo-labeled MR-to-Text pairs for further fine-tuning. However, some self-augmented data may be noisy or uninformative for the model to learn from. In this work, we propose a two-phase self-augmentation procedure to generate high-quality pseudo-labeled MR-to-Text pairs: the first phase selects the most informative MRs based on model’s prediction uncertainty; with the selected MRs, the second phase generates accurate responses by aggregating multiple perturbed latent representations from each MR. Empirical experiments on two benchmark datasets, FewShotWOZ and FewShotSGD, show that our method generally outperforms existing self-training methods on both automatic and human evaluations.
BibTeX
@inproceedings{du2022self,
  author = {Wanyu Du and Hanjie Chen and Yangfeng Ji},
  title = {{Self-training with Two-phase Self-augmentation for Few-shot Dialogue Generation}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2022},
  url = {https://openreview.net/forum?id=oPKlmzQeH7},
}
Figure