How Well Can a Genetic Algorithm Fine-tune Transformer Encoders? A First Approach

Vicente Ivan Sanchez Carmona, Shanshan Jiang, Bin DongNAACL 2024

TL;DR

This paper explores using Genetic Algorithms (GAs) to fine-tune Transformer encoders, a task traditionally dominated by gradient descent methods like Adam. While GAs are slower and yield lower accuracy, they surprisingly excel at leveraging knowledge from multiple pre-training datasets, suggesting a novel avenue for model optimization.

What Problem Does This Paper Address?

The field of Natural Language Processing (NLP) heavily relies on Transformer models, which are typically fine-tuned using gradient descent algorithms like Adam. A common assumption is that pre-training a Transformer on multiple datasets leads to superior performance compared to single-dataset pre-training, as it supposedly imbues the model with broader linguistic and world knowledge. However, this paper questions whether Adam is truly effective at exploiting this multi-dataset knowledge. The authors propose investigating Genetic Algorithms (GAs) as an alternative fine-tuning mechanism, exploring their ability to both fine-tune Transformers and, crucially, to better recombine knowledge from diverse pre-training sources.

Methods

The core of this research lies in comparing two distinct approaches for fine-tuning Transformer encoders:

  • Adam: The standard gradient descent optimizer, used as a baseline.
  • Genetic Algorithm (GA): Specifically, an Eclectic Genetic Algorithm (EGA) variant. This algorithm operates on a population of Transformer encoders, iteratively improving them through a cycle of:
  • Fitness Evaluation: Assessing encoder performance (accuracy) on a validation set.
  • Crossover: Combining "genetic material" (weight vectors) from parent encoders to create offspring. The paper highlights the simulated-binary crossover as a key component for knowledge recombination.
  • Mutation: Randomly altering weights in offspring to introduce diversity.

The GA is applied to fine-tune encoders that were either pre-trained on a single dataset (WikiText-103 or 1-Billion-Word) or on both.

Datasets

  • Pre-training Datasets:
    • WikiText-103
    • 1-Billion-Word (lm1b)
  • Fine-tuning Datasets (Sentiment Analysis):
    • SST-2
    • IMDB
    • Yelp

The sentiment analysis datasets were chosen for their interpretability, yielding binary accuracy scores.

Key Insights and Ideas

  • GA's Crossover as a Knowledge Recombiner: The paper strongly suggests that the GA's crossover operator is instrumental in its ability to effectively combine knowledge from different pre-training datasets. This is a novel perspective on how GAs might offer advantages beyond traditional gradient-based methods.
  • Surprising Performance Gap in Knowledge Exploitation: While Adam is the go-to for fine-tuning, the GA demonstrates a superior ability to leverage knowledge from multiple pre-training sources, achieving significantly larger performance gains than Adam in this specific scenario.
  • The "Ceiling Effect" Hypothesis: The authors cautiously suggest that Adam's seemingly modest gains from multi-dataset pre-training might be due to a "ceiling effect." This means the models might already be close to their maximum capability on the given task, leaving less room for improvement from additional pre-training data.
  • Dataset Interactions Can Be Complex: For the IMDB dataset, fine-tuning on a single pre-training dataset (wiki or lm1b) actually yielded better results than using both. This challenges the straightforward assumption that more pre-training data is always better and hints at complex interactions between datasets and tasks.

Key Results

  • Performance vs. Adam:
    • Accuracy: GAs achieved lower accuracy scores than Adam, with a drop of up to 28 points in some cases.
  • Training Time: GAs were significantly slower, taking up to 46 times longer than Adam.
  • Knowledge Recombination:
    • Adam: Fine-tuning with Adam on two pre-training datasets yielded an average gain of 0.55 accuracy points compared to single-dataset pre-training.
  • GA: Fine-tuning with GA on two pre-training datasets yielded an average gain of 1.65 accuracy points, which is approximately 3 times Adam's gain.
  • Relative Increase: The GA showed a remarkable relative increase in performance gain of up to 1540% for the Yelp dataset compared to Adam.
  • IMDB Anomaly: For the IMDB dataset, single pre-training datasets (wiki or lm1b) outperformed mixed pre-training for both Adam and GA. The GA showed a 598% relative increase in performance gain over Adam for IMDB when comparing mixed vs. wiki pre-training.

Limitations and Future Work

  • Limitations:
    • Computational Cost: GAs are significantly slower and more resource-intensive than Adam.
  • Lower Accuracy: GAs generally resulted in lower fine-tuning accuracy scores.
  • Ceiling Effect: The observed gains from multi-dataset pre-training might be limited by the task's inherent difficulty or the model's capacity, potentially masking the true potential of multi-dataset learning.
  • Specific Model/Task: The study focused on a particular Transformer encoder variant and sentiment analysis tasks.
  • Future Work:
    • More Complex GAs and Hybrids: Explore more advanced GA variants or hybrid approaches that combine the strengths of Adam (speed, high scores) and GAs (knowledge recombination).
  • Broader NLP Models: Test these methods on more complex models like BERT.
  • Diverse Datasets: Experiment with other pre-training datasets (e.g., BookCorpus, C4) and downstream tasks.
  • Deeper Analysis: Investigate the specific reasons behind the IMDB anomaly and the complex interactions between datasets.

Summary updated

BibTeX
@inproceedings{carmona2024how,
  author = {Vicente Ivan Sanchez Carmona and Shanshan Jiang and Bin Dong},
  title = {{How Well Can a Genetic Algorithm Fine-tune Transformer Encoders? A First Approach}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2024},
}

Loading related content...

Figure