Rational LAMOL: A Rationale-based Lifelong Learning Framework

Kasidis Kanwatchara, Thanapapas Horsuwan, Piyawat Lertvittayakumjorn, Boonserm Kijsirikul, Peerapon VateekulACL 2021

TL;DR

This paper introduces Rational LAMOL, a novel framework for lifelong learning in NLP that tackles catastrophic forgetting by identifying and freezing critical components of transformer models. By leveraging human or automatically generated rationales, Rational LAMOL significantly improves performance and robustness compared to existing methods, paving the way for more adaptable and persistent language models.

What Problem Does This Paper Address?

Lifelong learning (LL) aims to equip AI models with the ability to continuously learn from new data and tasks without forgetting previously acquired knowledge. This is crucial for real-world applications where data streams are dynamic and diverse. However, a major hurdle in LL, particularly in Natural Language Processing (NLP), is "catastrophic forgetting." This phenomenon causes models to drastically degrade performance on older tasks when trained on new ones, essentially "forgetting" what they learned. Existing LL approaches, while showing promise, often struggle to bridge the performance gap with multi-task learning (the theoretical upper bound) and can be computationally expensive. This paper addresses the need for a more effective and efficient LL framework that can mitigate catastrophic forgetting in NLP models.

Methods

The core of this work is Rational LAMOL, an extension of the LAMOL (Language Modeling for Lifelong Language Learning) framework. LAMOL itself frames all NLP tasks as question answering (QA) and uses its language modeling capability to generate pseudo-samples of old tasks to refresh knowledge. Rational LAMOL enhances this by introducing two key components:

  • Critical Component Identification (CCI): This algorithm identifies the "most plastic" components (layers or attention heads) in a transformer model that are most susceptible to forgetting when a new task is introduced. It achieves this by comparing attention maps from the model before and after training on a new task, using rationales to guide the identification of components that are most sensitive to changes.
  • Critical Freezing: Once the critical components are identified, they are "frozen" – their weights are not updated during training on the new task. This prevents them from being overwritten, thereby preserving learned knowledge.

A significant innovation is the use of unsupervised rationale generation (using the InvRat framework) as a substitute for human-annotated rationales. This allows Rational LAMOL to be applied to any NLP dataset, removing the dependency on costly human annotations.

Datasets

The experiments were conducted on three English text classification datasets from the ERASER benchmark:

  • BoolQ: Yes/no questions based on Wikipedia passages.
  • Movie Reviews: Sentiment analysis of movie reviews.
  • SciFact: Identifying whether scientific claims are supported or refuted by evidence abstracts.

The paper evaluated Rational LAMOL on all six permutations of task orders for these three datasets.

Key Insights and Ideas

  • Rationales as a Key to Model Understanding: The paper highlights that rationales, which explain why a model makes a certain prediction, are crucial for understanding and mitigating catastrophic forgetting. By aligning model attention with these rationales, we can pinpoint vulnerable parts of the model.
  • Freezing is More Than Just Regularization: While component freezing is a known technique, Rational LAMOL demonstrates its effectiveness in LL by specifically targeting plastic components identified through rationales, leading to more robust performance than generic freezing strategies.
  • Unsupervised Rationales are Surprisingly Effective: The ability to use automatically generated rationales (from InvRat) to achieve competitive performance is a major breakthrough. This significantly broadens the applicability of the framework to real-world scenarios where human annotations are scarce.
  • Task Order Matters, but Rational LAMOL is More Resilient: The experiments show that the order in which tasks are presented can significantly impact performance. However, Rational LAMOL demonstrates greater resilience to these order variations compared to the baseline LAMOL.

Key Results

  • Rational LAMOL (block-level freezing) outperformed the original LAMOL by an average of 1.83% accuracy across six task permutations, with a lower standard deviation.
  • Using unsupervised rationale generation (Gen-Rational LAMOL block) also yielded competitive results, achieving an average improvement of 2.67% accuracy over the original LAMOL.
  • The framework achieved performance close to the "brute force" upper bound, suggesting that critical freezing is a highly effective strategy.
  • The runtime overhead for Rational LAMOL was manageable, approximately 2.4 times that of vanilla LAMOL, which is significantly less than brute-force methods.

Limitations and Future Work

  • Human vs. Machine Rationale Alignment: The authors acknowledge that there can be incompatibilities between human rationales and machine attention scores, which might lead to sub-optimal layer/head choices. This is an area for future research.
  • Single-Model Analysis: The current CCI algorithm analyzes plasticity between two consecutive models ($M_i$ and $M_{i+1}$). Future work could involve analyzing plasticity within a single model over time.
  • Head-Level vs. Block-Level Freezing: While both block-level and head-level freezing showed improvements, the paper found no conclusive evidence for one being significantly better than the other. Further investigation into optimal granularity is suggested.
  • Generalizability to Other Architectures: The paper notes that while Rational LAMOL is designed for transformer-based models, its potential applications to other attention-based architectures suffering from catastrophic forgetting warrant further exploration.

Summary updated

BibTeX
@inproceedings{kanwatchara2021rational,
  author = {Kasidis Kanwatchara and Thanapapas Horsuwan and Piyawat Lertvittayakumjorn and Boonserm Kijsirikul and Peerapon Vateekul},
  title = {{Rational LAMOL: A Rationale-based Lifelong Learning Framework}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2021},
}
Figure