Introduction: The Challenge of Catastrophic Forgetting
Large language models (LLMs) like GPT and BERT have revolutionized natural language processing, yet they often struggle with "strict" mathematical reasoning and the precise handling of numerals. While fine-tuning on specialized datasets seems like a straightforward solution, it frequently triggers catastrophic forgetting. In this phenomenon, the weights essential for a model's original task (e.g., language understanding) are overwritten to accommodate a new task (e.g., arithmetic), leading to a total loss of the original skill.
This paper investigates this "math vs. language" trade-off using a DistilBERT model. The authors observed that standard fine-tuning on arithmetic tasks caused the model to completely lose its ability to judge grammatical correctness, even as it became proficient in math. To address this, they propose an information-theoretic approach to inject new skills while preserving existing ones.
Methodology: Information-Theoretic Skill Injection
The researchers developed a two-pronged strategy to overcome the barriers of skill injection:
1. Information-Theoretic Analysis
Using Fisher Information, the authors identified which specific parameters in the model are most vital for linguistic tasks. By calculating the sensitivity of parameters, they discovered that linguistic and arithmetic tasks effectively "live" in different regions of the model's parameter space. This sensitivity analysis allows for the identification of a subset of weights that must be protected to maintain linguistic performance.
2. Elastic Weight Consolidation (EWC)
To operationalize these insights, the authors implemented Elastic Weight Consolidation (EWC). Unlike standard fine-tuning, EWC introduces a penalty to the training loss function. If the optimization process attempts to significantly alter a weight flagged by the Fisher Information matrix as important for language, the penalty increases. This constraint forces the model to find a mathematical solution within the "neighborhood" of its original linguistic capabilities, effectively finding a "sweet spot" in the parameter space.
Visualizing the Parameter Space
To validate their approach, the authors employed t-SNE projections to map the high-dimensional parameter space. These visualizations confirmed that models trained solely on arithmetic drift significantly away from the clusters formed by models trained on linguistic tasks.
Key Results and Discussion
The effectiveness of the EWC-based approach was evaluated against a base DistilBERT and a standard math-tuned version across several benchmarks:
- Mathematical Proficiency: The EWC model achieved the highest arithmetic accuracy, with a log Root Mean Square Error (ln RMSE) of 0.4360, a vast improvement over the base model's 3.5367.
- Linguistic Preservation: The most striking result was in grammatical correctness (CoLA). While standard math-tuning caused the CoLA score to plummet from 0.4827 to 0.0000, the EWC model maintained a score of 0.4193.
- Task Consistency: Across other GLUE tasks like MNLI (logic) and SST-2 (sentiment), the EWC model remained within a few percentage points of the base model, whereas standard fine-tuning led to significant degradation.
Conclusion and Future Directions
This research demonstrates that the trade-off between mathematical and linguistic skills in LLMs is not inevitable. By using information-theoretic constraints like EWC, it is possible to inject specialized skills into a model while preserving its foundational capabilities.
Future work will look to expand this framework beyond simple addition and subtraction to include more complex operations like multiplication and division, as well as handling decimal-point numerals. Ultimately, the authors aim to apply this methodology to a broader range of non-linguistic skills, paving the way for more versatile and robust multi-task language models.
Summary updated