Learning Non-linguistic Skills without Sacrificing Linguistic Proficiency

Mandar Sharma, Nikhil Muralidhar, Naren Ramakrishnan

Abstract

A significant challenge in developing Large Language Models (LLMs) is the trade-off between specialized skill acquisition and general linguistic intelligence. Fine-tuning models on non-linguistic tasks, such as arithmetic, often leads to "catastrophic forgetting" of core language capabilities. This paper introduces Skill-LM, a framework designed to learn arithmetic reasoning without sacrificing linguistic proficiency. By employing Elastic Weight Consolidation (EWC) for information-theoretic weight anchoring and a novel magnitude-aware regression loss, Skill-LM outperforms existing models using significantly less data and no synthetic linguistic samples.

Introduction

The integration of quantitative reasoning into Natural Language Processing (Math-NLP) aims to provide LLMs with calculator-like precision. However, traditional fine-tuning on niche tasks often overwrites weights essential for grammar or sentiment analysis. Previous solutions, like GenBERT, relied on computationally expensive "rehearsal" methods—mixing millions of synthetic language samples into training data. Skill-LM addresses the root cause by identifying "load-bearing" linguistic weights and treating numbers as magnitudes rather than mere text tokens.

Conceptual overview of Skill-LM: balancing linguistic and non-linguistic task proficiency within the model parameter space.

Methodology

The Skill-LM framework utilizes two primary interventions to maintain a balance between new skills and existing knowledge:

1. Linguistic Retention via Elastic Weight Consolidation (EWC)

The researchers first calculate the Fisher Information Index of the base model on a general corpus (WikiText). This index identifies which specific parameters are critical for linguistic tasks. During arithmetic training, EWC penalizes significant changes to these vital weights, effectively "anchoring" the model to its linguistic foundations.

2. Magnitude-Aware Learning ($L_{REG}$)

Standard Cross-Entropy loss treats numbers as categorical tokens, ignoring their numerical relationships. Skill-LM introduces a Regression Loss ($L_{REG}$) that forces the model to learn actual numerical values. This helps the model understand the "number line," leading to better generalization on out-of-domain (OOD) data.

Key Insights: Task-Specific Forgetting

Analysis using Fisher Information revealed that forgetting is not uniform across all linguistic skills. Arithmetic training overlaps significantly with parameters used for grammatical correctness (CoLA) but shows minimal overlap with those used for pronoun resolution (WNLI). This explains why certain linguistic capabilities are more vulnerable during specialized fine-tuning.

Fisher Information Sensitivity across individual model parameters for various GLUE tasks in Encoder Self-Attention Layers 1 (left) and 12 (right).

Results

Skill-LM demonstrates remarkable efficiency and robustness:

  • Data Efficiency: It matches state-of-the-art performance (GenBERT) using 75% less arithmetic training data and zero synthetic language samples.
  • Language Preservation: On the CoLA grammar task, while a naively fine-tuned BERT's score dropped from 0.59 to 0.08, Skill-LM maintained a score of 0.58.
  • Mathematical Generalization: In tests involving numbers larger than those in the training set, Skill-LM achieved 19.44% accuracy, compared to 1.32% for GenBERT. Qualitative results showed that Skill-LM's errors were typically within the correct order of magnitude.

Conclusion

Skill-LM provides a scalable path for enhancing LLMs with specialized non-linguistic skills while preserving their general-purpose utility. By focusing on the locality of parameterization and magnitude-aware learning, the framework avoids the pitfalls of catastrophic forgetting without the overhead of massive synthetic data rehearsal.

Summary updated

Abstract

The field of Math-NLP has witnessed significant growth in recent years, motivated by the desire to expand LLM performance to the learning of non-linguistic notions (numerals, and subsequently, arithmetic reasoning). However, non-linguistic skill injection typically comes at a cost for LLMs: it leads to catastrophic forgetting of core linguistic skills, a consequence that often remains unaddressed in the literature. As Math-NLP has been able to create LLMs that can closely approximate the mathematical skills of a grade-schooler or the arithmetic reasoning skills of a calculator, the practicality of these models fail if they concomitantly shed their linguistic capabilities. In this work, we take a closer look into the phenomena of catastrophic forgetting as it pertains to LLMs and subsequently offer a novel framework for non-linguistic skill injection for LLMs based on information theoretic interventions and skill-specific losses that enable the learning of strict arithmetic reasoning. Our model outperforms the state-of-the-art both on injected non-linguistic skills and on linguistic knowledge retention, and does so with a fraction of the non-linguistic training data (1/4) and zero additional synthetic linguistic training data.
BibTeX
@inproceedings{sharma2023learning,
  author = {Mandar Sharma and Nikhil Muralidhar and Naren Ramakrishnan},
  title = {{Learning Non-linguistic Skills without Sacrificing Linguistic Proficiency}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2023},
  doi = {10.18653/v1/2023.acl-long.340},
}
Figure