Friend-training: Learning from Models of Different but Related Tasks

Mian Zhang, Lifeng Jin, Linfeng Song, Haitao Mi, Xiabing Zhou, Dong Yu

Abstract

Friend-training is a novel semi-supervised learning framework designed to mitigate the "confirmation bias" inherent in standard self-training. By leveraging models trained on different but linguistically related tasks—such as Dialogue Rewriting (DR) and Conversational Semantic Role Labeling (CSRL)—the framework uses a "Translation Matcher" to validate pseudo-labels across task boundaries. This cross-task supervision significantly enhances performance in few-shot and out-of-domain scenarios, outperforming traditional self-training and other state-of-the-art semi-supervised methods like Mean Teacher and Cross Pseudo Supervision.

Introduction: The Challenge of Self-Training

The "data hunger" of modern NLP models is a significant bottleneck. While unlabeled text is abundant, turning it into useful training data typically involves self-training, where a model labels the data itself (pseudo-labeling) and then retrains on its own guesses. However, this often leads to "confirmation bias" or "gradual drift"—if the model makes an initial mistake, it reinforces that error during retraining, creating a cycle of noise.

Conceptual overview of cross-task supervision between Dialogue Rewriting and Conversational Semantic Role Labeling.

Friend-training addresses this by arguing that different NLP tasks often share underlying linguistic "DNA." For example, resolving pronouns in a dialogue (Dialogue Rewriting) and identifying semantic roles (CSRL) both rely on understanding coreference. As shown in Figure 1, if a rewriter correctly identifies a referent, it can provide a corrective signal to a CSRL model that might have misidentified the argument.

Methodology

The Friend-training (FDT) framework is an iterative process consisting of two primary novel components:

  1. Translation Matcher ($\mathcal{M}$): This component acts as a bridge between "friend" tasks. Since different tasks produce different output types (e.g., a rewritten sentence vs. a set of semantic tags), the matcher maps these outputs into a shared "translation space." In the case study, a standard sentence-level SRL parser analyzes the output of the Dialogue Rewriter. If the rewriter's output and the CSRL model's output agree on the semantic roles, the pseudo-label is considered high-quality.
  2. Augmented Selector: Standard self-training relies on a model's internal softmax confidence. The Augmented Selector combines this internal confidence with the matching score from the friend task. This ensures that a model isn't just "confident and wrong"—it must also be validated by its "friend."

The mathematical core of this approach suggests that because the two models are trained on different tasks, their error patterns are likely independent. It is highly improbable that two different models would make the exact same mistake in a way that perfectly aligns in the translation space, making "agreement" a much stronger signal than "confidence" alone.

Experimental Setup

The researchers evaluated FDT on two dialogue-heavy tasks:

  • Conversational Semantic Role Labeling (CSRL): Trained on DuConv and tested on a new, manually annotated out-of-domain dataset, WeiboCSRL.
  • Dialogue Rewriting (DR): Trained on REWRITE and evaluated on the RESTORATION dataset.
  • Unlabeled Data: 20,000 dialogues sampled from the LCCC-base corpus.

Key Results and Insights

Friend-training consistently outperformed standard self-training (SST) and several state-of-the-art semi-supervised methods.

Performance comparison of Friend-training (FDT) against baselines on CSRL and DR test sets.

Performance Gains

  • Few-Shot Learning: The gains were most dramatic in low-data regimes. As shown in Figure 6, FDT achieved a 4.51 point F1 improvement in CSRL and a 3.15 point WER reduction in DR compared to SST when starting with only 100 labeled examples.
  • Domain Generalization: On the out-of-domain WeiboCSRL task, FDT improved the F1 score by 3.16 points over standard self-training.
  • Robustness: FDT showed a steeper improvement curve as the quality of the base models increased, proving it leverages unlabeled data more efficiently than its competitors.

The Power of Independence

The study found that "co-updating"—letting both models learn and improve together—generally outperforms using a single fixed, high-quality teacher. Furthermore, the larger the shared "translation space" (the more complex the information they agree on), the lower the probability of erroneous labels slipping through.

Limitations and Future Work

Despite its success, Friend-training currently requires human experts to identify "friend" tasks and manually design the translation functions. Future research will focus on automated ways to discover these relationships using task grouping or similarity metrics. Additionally, the authors see potential in applying this framework to cross-modal tasks, such as pairing vision and language models as "friends."

Summary updated

BibTeX
@inproceedings{zhang2023friend,
  author = {Mian Zhang and Lifeng Jin and Linfeng Song and Haitao Mi and Xiabing Zhou and Dong Yu},
  title = {{Friend-training: Learning from Models of Different but Related Tasks}},
  booktitle = {EACL},
  year = {2023},
  url = {https://openreview.net/forum?id=VtvPXumFdiu},
}
Figure