An Investigation of Suitability of Pre-Trained Language Models for Dialogue Generation - Avoiding Discrepancies

Yan Zeng, Jian-Yun NieACL 2021 · DOI

TL;DR

This paper investigates how different pre-trained language model architectures (Transformer-ED, Transformer-Dec, Transformer-MLM, Transformer-AR) perform in dialogue generation, especially when fine-tuned on limited data. They introduce "discrepancy" concepts to explain performance differences and propose methods to mitigate these issues, showing improved results.

What Problem Does This Paper Address?

The field of dialogue generation has seen great success by leveraging large pre-trained language models (LMs). However, there's a debate on which pre-trained LM architectures are best suited for this task. Some argue for decoder-only models (like GPT-2) that naturally align with left-to-right generation, while others champion models pre-trained with bidirectional attention (like BERT) that can capture richer context. This paper aims to settle this by systematically comparing four common frameworks (Transformer-ED, Transformer-Dec, Transformer-MLM, Transformer-AR) across various dataset sizes. A key challenge they highlight is that the effectiveness of pre-trained LMs can be hindered by "discrepancies" between how the model was pre-trained and how it's fine-tuned or used for generation, especially when fine-tuning data is scarce.

Methods

The authors compare four established frameworks for dialogue generation:

  • Transformer-ED: An encoder-decoder architecture.
  • Transformer-Dec: A decoder-only architecture using left-to-right attention (e.g., GPT-2).
  • Transformer-MLM: A decoder-only architecture using bidirectional attention on the source and masked language modeling (MLM) for the target.
  • Transformer-AR: Similar to Transformer-MLM but uses an autoregressive objective for the target.

They introduce two key concepts to analyze model performance:

  • Pretrain-Finetune Discrepancy: Differences in architecture or loss function between pre-training and fine-tuning.
  • Finetune-Generation Discrepancy: Differences in how the model is used during fine-tuning versus inference.

To address these discrepancies, they propose two methods for Transformer-MLM:

  • PF-free: Reduces pretrain-finetune discrepancy by allowing bidirectional attention on the target side with a time-step interval.
  • FG-free: Eliminates finetune-generation discrepancy by modifying the training objective to better match inference.

Datasets

The paper uses three large-scale unlabeled dialogue datasets:

  • Twitter Dialogue Corpus: ~2 million samples.
  • Reddit Conversational Corpus: A 3-turn conversational dataset.
  • Ubuntu Dialogue Corpus V2.0: Conversations extracted from Ubuntu chat logs.

They evaluate performance using both large-scale data and a smaller subset of 100K samples for each dataset to study the impact of data size on model discrepancies.

Key Insights and Ideas

  • Decoder-only architectures shine: On large datasets, Transformer-Dec and Transformer-AR (decoder-only) outperform Transformer-ED (encoder-decoder).
  • Discrepancies matter, especially with limited data: The performance gap between models with high discrepancies and those with low discrepancies widens significantly when fine-tuning on small datasets. This suggests that careful consideration of pre-training/fine-tuning alignment is crucial.
  • Left-to-right vs. Bidirectional attention trade-offs: Left-to-right attention (Trans-Dec) leads to more diverse responses, while bidirectional attention (Trans-MLM/AR) models context better and yields higher BLEU/CIDEr scores.
  • MLM vs. Autoregressive objectives: While MLM can be effective, the autoregressive objective (Trans-AR) seems more training-efficient for large-scale dialogue generation.
  • Mitigating discrepancies improves performance: The proposed PF-free and FG-free methods for Transformer-MLM demonstrate that reducing discrepancies leads to better results, particularly FG-free which significantly boosts performance.

Key Results

  • Large-scale data: Trans-Dec and Trans-AR achieve the best results. Trans-AR generates responses most similar to ground-truth, while Trans-Dec produces the most diverse outputs.
  • Small-scale data: Trans-Dec is the most appropriate, showing that models with fewer discrepancies (like Trans-Dec) are more robust when data is limited.
  • Proposed methods: FG-free significantly outperforms vanilla Trans-MLM across automatic metrics and achieves comparable human evaluation scores to Trans-Dec. PF-free shows some improvement but less consistently.
  • Human Evaluation: Trans-Dec and Trans-AR are rated as generating the most appropriate responses on large-scale data.

Limitations and Future Work

  • Limited data impact: The paper emphasizes that while large-scale pre-training can mitigate the impact of discrepancies, they remain a significant factor when fine-tuning on smaller datasets.
  • Future work: The authors suggest that further investigation into architectural appropriateness and discrepancy reduction is warranted for effective pre-trained LM usage in dialogue generation. They also note that while their proposed methods show improvement, there's still room for further optimization.

Summary updated

BibTeX
@inproceedings{zeng2021investigation,
  author = {Yan Zeng and Jian-Yun Nie},
  title = {{An Investigation of Suitability of Pre-Trained Language Models for Dialogue Generation - Avoiding Discrepancies}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2021},
  doi = {10.18653/v1/2021.findings-acl.393},
}
Figure