Heterogeneous-Branch Collaborative Learning for Dialogue Generation

Yiwei Li, Shaoxiong Feng, Bin Sun, Kan Li

TL;DR

Training lightweight dialogue agents typically requires distilling knowledge from a massive, pre-trained teacher model, while teacher-free "collaborative learning" often fails because student branches quickly collapse into identical representations. This paper introduces Collaborative Dialogue Learning (CDL), which guides different student branches to specialize in distinct conversational attributes (coherence, informativeness, and specificity) and uses a novel orthogonal negative distillation mechanism to diversify hidden representations without destroying shared linguistic fundamentals.

What Problem Does This Paper Address?

Deploying high-quality neural conversational models on resource-constrained platforms (such as mobile devices) demands compact, parameter-efficient architectures. Standard knowledge distillation (KD) compresses models effectively, but it requires pre-training a cumbersome teacher model first—an expensive, multi-stage process.

Online knowledge distillation (or collaborative learning) removes the teacher by letting multiple compact student branches learn together simultaneously. However, because these branches share identical objectives and training data, they suffer from branch homogeneity: branches rapidly converge to nearly identical representations, causing early performance saturation. In dialogue generation, where outputs must balance distinct qualitative attributes like coherence, informativeness, and vocabulary specificity, this homogenization is especially detrimental.

Methods

The authors propose CDL, a collaborative learning framework featuring a single master branch and multiple auxiliary branches:

[Full Dataset] --------> Master Branch <---(Positive Distillation)---> [Auxiliary Branches]
                              |                                                ^
                              |                                      (Negative Distillation:
[Attribute Subsets] ---> Auxiliary Branches                           Prediction & Hidden Space)
  1. Attribute-Specific Specialization: Rather than training all branches on identical data, the raw dataset is split into targeted sub-sets using established scoring metrics:
  2. Coherence ($C$): Measures keyword co-occurrence (nPMI) and context-response semantic cosine similarity.
  3. Informativeness ($I$): Evaluates utterance entropy to filter out bland, generic responses.
  4. Specificity ($S$): Uses inverse document frequency (IDF) scoring to prioritize rare, informative words. Each auxiliary branch trains on the top-scoring 70% subset for its designated attribute, while the master branch trains on the entire dataset.

  5. Dual Knowledge Distillation:

  6. Bidirectional Positive Distillation (PD): Soft probability distributions are distilled back and forth between the master branch and each auxiliary branch at the output layer. The master absorbs specialized attribute knowledge, while auxiliary branches benefit from the master's generalized context.
  7. Negative Distillation (ND) across Auxiliaries: Auxiliaries are pushed apart using a soft unlikelihood loss on output probabilities, actively discouraging branches from making identical predictions.
  8. Orthogonal Negative Distillation: Blindly repelling intermediate hidden states ruins the shared grammatical and syntactic knowledge required to form fluent sentences. To solve this, the authors project branch $A$'s decoder hidden states onto the orthogonal subspace of branch $B$'s hidden states ($H_L = (I - H_B (H_B^T H_B)^{-1} H_B^T) H_A$). Negative distillation is applied strictly to this orthogonal projection, repelling only attribute-biased features while leaving common language representations intact.

Datasets

The authors evaluated CDL on two standard open-domain dialogue benchmarks (no new datasets were introduced):

  • DailyDialog: 68,066 training, 6,820 validation, and 6,841 test pairs representing daily human conversations.
  • OpenSubtitles: 200,000 training, 20,000 validation, and 10,000 test context-response pairs extracted from movie dialogues.

Key Insights and Ideas

  • Dialogue Attributes as a Natural Inductive Bias for Diversity: In classification tasks, diversity heuristics can feel arbitrary. In dialogue, conversational attributes (e.g., informativeness vs. coherence) offer an intuitive way to assign distinct functional roles to different network branches.
  • Protecting Foundational Syntax via Orthogonal Projection: Auxiliary branches need to agree on language fundamentals (grammar, common sentence flow) while disagreeing on stylistic attributes. Projecting representations into an orthogonal subspace isolates stylistic differences, preventing negative distillation from corrupting fundamental language competence.
  • Outperforming Heavy Teacher Models: Collaborative training between compact peers achieved higher response diversity and lower KL divergence than standard offline distillation from a much larger, pre-trained base model.

Key Results

  • Significant Diversity Gains: On DailyDialog, CDL pairing Informativeness and Specificity (CDL-IS) achieved a Dist-2 of 0.1081 and Dist-3 of 0.2143, vastly outperforming the baseline small Transformer (0.0345 / 0.0748) and collaborative baselines like DML (0.0669 / 0.1296) and OKDDip (0.0581 / 0.1168).
  • Beating Traditional KD: CDL-IS surpassed a traditional KD model distilled from a Base Transformer teacher across lexical diversity (Dist-2 of 0.1081 vs. 0.0564) and KL divergence to ground truth (0.32 vs. 0.47).
  • Branch Diversity: Inter-branch Euclidean distance ($L_2$) measured 0.308 for CDL versus 0.141–0.198 for standard collaborative baselines, confirming that branch homogenization was successfully prevented.
  • Human Evaluations: Human judges strongly favored CDL responses over vanilla Transformers (82% win rate) and consistently preferred CDL over peer-collaborative baselines (43% to 50% win rates vs. 5% to 11% loss rates).

Limitations and Future Work

  • Branch Scalability: When scaling from 3 to 4 branches (incorporating $C$, $S$, and $I$ simultaneously), performance slightly dropped. The authors observed that adding more auxiliary peers generated excessive negative distillation signals, which overpowered the stabilizing positive distillation from the master branch.
  • Dynamic Loss Weighting: The balance between positive and negative distillation weights currently remains fixed; future work will focus on adaptive schedules to automatically weight positive vs. negative distillation losses during training.

Summary updated

Abstract

With the development of deep learning, advanced dialogue generation methods usually require a greater amount of computational resources. One promising approach to obtaining a high-performance and lightweight model is knowledge distillation, which relies heavily on the pre-trained powerful teacher. Collaborative learning, also known as online knowledge distillation, is an effective way to conduct one-stage group distillation in the absence of a well-trained large teacher model. However, previous work has a severe branch homogeneity problem due to the same training objective and the independent identical training sets. To alleviate this problem, we consider the dialogue attributes in the training of network branches. Each branch learns the attribute-related features based on the selected subset. Furthermore, we propose a dual group-based knowledge distillation method, consisting of positive distillation and negative distillation, to further diversify the features of different branches in a steadily and interpretable way. The proposed approach significantly improves branch heterogeneity and outperforms state-of-the-art collaborative learning methods on two widely used open-domain dialogue datasets.
BibTeX
@inproceedings{li2023heterogeneous,
  author = {Yiwei Li and Shaoxiong Feng and Bin Sun and Kan Li},
  title = {{Heterogeneous-Branch Collaborative Learning for Dialogue Generation}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2023},
  url = {https://openreview.net/forum?id=LP0WK5VB1Y},
}

Loading related content...

Figure