Heterogeneous-Branch Collaborative Learning for Dialogue Generation

Yiwei Li, Shaoxiong Feng, Bin Sun, Kan Li

Abstract

With the development of deep learning, advanced dialogue generation methods usually require a greater amount of computational resources. One promising approach to obtaining a high-performance and lightweight model is knowledge distillation, which relies heavily on the pre-trained powerful teacher. Collaborative learning, also known as online knowledge distillation, is an effective way to conduct one-stage group distillation in the absence of a well-trained large teacher model. However, previous work has a severe branch homogeneity problem due to the same training objective and the independent identical training sets. To alleviate this problem, we consider the dialogue attributes in the training of network branches. Each branch learns the attribute-related features based on the selected subset. Furthermore, we propose a dual group-based knowledge distillation method, consisting of positive distillation and negative distillation, to further diversify the features of different branches in a steadily and interpretable way. The proposed approach significantly improves branch heterogeneity and outperforms state-of-the-art collaborative learning methods on two widely used open-domain dialogue datasets.

Introduction

Open-domain dialogue models are often over-parameterized, making them difficult to deploy on resource-constrained edge hardware. While Knowledge Distillation (KD) offers a path toward model compression, traditional offline KD requires a pre-trained, high-capacity "teacher" model, which is computationally expensive to produce. Collaborative learning (or online KD) offers a teacher-free alternative where multiple student branches learn from one another simultaneously.

However, collaborative learning faces a critical challenge: branch homogeneity. Because all student branches typically share the same training objective and data, they tend to converge toward identical representations. This lack of diversity nullifies the benefits of peer learning. This paper introduces the Collaborative Dialogue Learning (CDL) framework to solve this by enforcing branch heterogeneity through attribute specialization and a novel dual-distillation mechanism.

Methodology

The CDL framework introduces two primary innovations designed to maintain diversity among student branches while improving the overall quality of the master model.

1. Attribute-Aware Auxiliary Branches

Rather than training all branches on the entire dataset with a uniform objective, the CDL framework splits the training data into subsets based on three key dialogue attributes:

  • Coherence: The logical fit between the response and the context.
  • Informativeness: The amount of new information provided in the response.
  • Specificity: The uniqueness and precision of the vocabulary used.

Each auxiliary branch specializes in one of these attributes, ensuring that different parts of the network develop distinct "expertise."

2. Dual Knowledge Distillation

To integrate these specialized insights, the framework employs a dual-distillation strategy:

  • Positive Distillation: This process transfers the specialized knowledge from the attribute-aware auxiliary branches into a central "Master Branch."
  • Negative Distillation: To prevent the branches from eventually merging into a single homogeneous state, the authors use negative distillation to push their representations apart in the feature space.
  • Orthogonal Projection: To ensure that negative distillation does not destroy the shared linguistic foundations (like basic grammar) that all branches must possess, the authors project the features of one branch into the orthogonal space of another. This amplifies the unique, biased features of each branch without interfering with common knowledge.

Experimental Setup

The model was evaluated on two benchmark datasets for open-domain dialogue:

  1. DailyDialog: High-quality human conversations regarding daily life.
  2. OpenSubtitles: A large-scale, diverse dataset of movie script dialogues.

Performance was measured using diversity metrics (Dist-n), word entropy (H-n), coherence (COH), and human evaluation.

Results and Analysis

Quantitative Performance

The CDL framework demonstrated significant improvements over existing collaborative learning baselines. On the DailyDialog dataset, CDL achieved a Dist-1 score of 0.0252, nearly doubling the performance of the OKDDip baseline (0.0141). Improvements in Word Entropy and Coherence metrics suggest that the attribute-specific training successfully transferred to the final generated output.

Branch Heterogeneity

The effectiveness of the negative distillation was confirmed by measuring the L2 distance between branch representations. The CDL model maintained a significantly higher L2 distance compared to standard collaborative methods, proving that the branches remained distinct throughout the training process.

Human Evaluation

In head-to-head matchups, human judges preferred responses generated by the CDL model over a standard Transformer baseline 82% of the time, citing better relevance and more engaging content.

Conclusion

The Heterogeneous-Branch Collaborative Learning framework successfully addresses the homogeneity trap in online knowledge distillation. By combining attribute-aware specialization with orthogonal negative distillation, the model achieves state-of-the-art performance without the need for a massive pre-trained teacher.

Future research will address the "diminishing returns" observed when scaling beyond three auxiliary branches. The authors hypothesize that as the number of branches increases, the negative distillation pressure may begin to dilute the positive knowledge transfer, suggesting a need for more sophisticated, dynamic weighting of distillation losses.

Summary updated

Abstract

With the development of deep learning, advanced dialogue generation methods usually require a greater amount of computational resources. One promising approach to obtaining a high-performance and lightweight model is knowledge distillation, which relies heavily on the pre-trained powerful teacher. Collaborative learning, also known as online knowledge distillation, is an effective way to conduct one-stage group distillation in the absence of a well-trained large teacher model. However, previous work has a severe branch homogeneity problem due to the same training objective and the independent identical training sets. To alleviate this problem, we consider the dialogue attributes in the training of network branches. Each branch learns the attribute-related features based on the selected subset. Furthermore, we propose a dual group-based knowledge distillation method, consisting of positive distillation and negative distillation, to further diversify the features of different branches in a steadily and interpretable way. The proposed approach significantly improves branch heterogeneity and outperforms state-of-the-art collaborative learning methods on two widely used open-domain dialogue datasets.
BibTeX
@inproceedings{li2023heterogeneous,
  author = {Yiwei Li and Shaoxiong Feng and Bin Sun and Kan Li},
  title = {{Heterogeneous-Branch Collaborative Learning for Dialogue Generation}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2023},
  url = {https://openreview.net/forum?id=LP0WK5VB1Y},
}
Figure