On the Interplay Between Fine-tuning and Composition in Transformers

Lang Yu, Allyson Ettinger

TL;DR

This paper investigates whether fine-tuning transformer language models on specific tasks can improve their ability to understand compositional meaning in phrases, beyond just recognizing individual words. The study found that fine-tuning on paraphrase or sentiment tasks generally failed to significantly boost compositional understanding, suggesting that current NLP datasets might not effectively probe or encourage this crucial linguistic capability.

What Problem Does This Paper Address?

Transformer-based language models like BERT and GPT have achieved remarkable success across various Natural Language Processing (NLP) tasks. However, recent research has raised concerns about their true understanding of language, particularly regarding "compositionality." Compositionality refers to a model's ability to combine the meanings of individual words to understand the meaning of a larger phrase or sentence. Previous work by Yu and Ettinger (2020) showed that while these models are good at recognizing word content, they struggle to demonstrate sophisticated compositional understanding, especially when lexical overlap between phrases is controlled. This paper aims to explore whether fine-tuning these models on tasks designed to require composition—like paraphrase identification with high word overlap and sentiment analysis of phrases—can bridge this gap and foster better compositional representations.

Methods

The researchers fine-tuned several pre-trained transformer models (BERT, RoBERTa, DistilBERT, XLNet, and XLM-RoBERTa) on two specific datasets: the Quora Question Pairs dataset from PAWS (PAWS-QQP) and the Stanford Sentiment Treebank (SST). After fine-tuning, they evaluated the models' phrasal representations using the same methodology as Yu and Ettinger (2020). This involved two main evaluation tasks:

  1. Similarity Correlation: Measuring how well model representations of phrase pairs align with human similarity judgments, using the BiRD dataset.
  2. Paraphrase Classification: Training a classifier to determine if two phrase representations are paraphrases, using the PPDB 2.0 dataset.

Crucially, both evaluation tasks were performed in "controlled" settings where lexical overlap between phrases was minimized or precisely controlled. This is to isolate the effect of composition from mere word matching. They also tested various ways of extracting phrase representations from the models (e.g., CLS token, average of token embeddings).

Datasets

  • Training Datasets:
    • PAWS-QQP (Paraphrase Adversaries from Word Scrambling - Quora Question Pairs): A dataset of sentence pairs with high lexical overlap, designed to be challenging for models that rely on superficial cues.
  • Stanford Sentiment Treebank (SST): A dataset containing phrases annotated with sentiment labels, intended to encourage compositional understanding of sentiment.
  • Evaluation Datasets:
    • BiRD (Bigram Relatedness Dataset): Used for similarity correlation tasks, containing bigram pairs with human-annotated similarity scores.
  • PPDB 2.0 (Paraphrase Database): Used for paraphrase classification tasks, providing paraphrase and non-paraphrase pairs.

Key Insights and Ideas

  • Fine-tuning doesn't magically fix composition: Despite selecting tasks that should encourage compositional understanding, fine-tuning had surprisingly little positive impact on the models' performance in controlled settings designed to test composition.
  • PAWS-QQP is a tricky dataset: The PAWS-QQP dataset appears to contain "spurious cues" (like the distance of word swaps) that allow models to achieve high accuracy without truly understanding compositional meaning. This highlights the need for caution when interpreting performance on NLP benchmarks.
  • SST offers a glimmer of hope: Fine-tuning on SST showed some localized, albeit small, improvements in compositional understanding for certain models and representation types (especially BERT's CLS token). This suggests that training on structured, hierarchical data with fine-grained labels might be more beneficial.
  • Lexical content still dominates: Even after fine-tuning, the models' ability to match human judgments on phrase similarity and paraphrase identification in controlled settings was still heavily influenced by word overlap, not true compositional understanding.

Key Results

  • PAWS-QQP Fine-tuning: Fine-tuning on PAWS-QQP generally did not improve compositional performance and sometimes even degraded it, despite models achieving high accuracy on the PAWS-QQP task itself.
  • SST Fine-tuning: Fine-tuning on SST showed minimal improvements in compositional tasks. However, for BERT, there were some notable, localized increases in similarity correlation, particularly in the CLS token representation at certain layers.
  • Controlled vs. Uncontrolled: The significant gap between performance on uncontrolled (high overlap) and controlled (low overlap) datasets persisted even after fine-tuning, reinforcing that models primarily rely on lexical cues. For instance, paraphrase classification accuracy on controlled datasets remained only marginally above chance.

Limitations and Future Work

  • Limitations:
    • The study found that fine-tuning on PAWS-QQP did not significantly improve compositional understanding, likely due to spurious cues in the dataset.
  • Improvements from SST fine-tuning were minimal and localized, suggesting that sentiment alone is a limited aspect of compositional meaning.
  • The presence of context words in sentences did boost performance slightly but did not fundamentally alter the trends observed with phrase-only inputs.
  • Future Work:
    • The authors suggest that training on datasets with "meaning-rich annotations" and diverse phrase structures could be a more promising direction for improving compositional understanding.
  • Further investigation into how specific model properties (layers, attention heads) interact with fine-tuning to produce improvements is needed.
  • Exploring fine-grained syntactic phrase distinctions in training data might confer more robust compositional enhancement.

Summary updated

BibTeX
@inproceedings{yu2021interplay,
  author = {Lang Yu and Allyson Ettinger},
  title = {{On the Interplay Between Fine-tuning and Composition in Transformers}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2021},
  url = {https://openreview.net/forum?id=wWMt4UZqvcA},
}
Figure