Catastrophic Interference in Connectionist Networks: Can It Be Predicted, Can It Be Prevented?

Robert M. FrenchNeurIPS 1993

Introduction

Neural networks often suffer from "catastrophic interference" (or catastrophic forgetting), a phenomenon where learning new information completely overwrites previously learned knowledge. This paper explores strategies to prevent this by reducing representation overlap in hidden layers. By employing techniques like prototype biasing, pretraining, and input orthogonalization, researchers demonstrate that it is possible to mitigate forgetting while preserving the network's crucial ability to generalize.

The Problem of Catastrophic Forgetting

In the early 1990s, connectionist networks (the precursors to modern deep learning) struggled immensely with sequential learning. Training a model on Task A and then moving to Task B typically resulted in the weight updates for Task B erasing the patterns learned for Task A.

The root cause lies in "distributed representations." While having many neurons participate in representing a single concept is excellent for generalization, it means new information inevitably overwrites the same neurons used for older information. The core challenge investigated in this workshop summary is whether we can force these representations to be more distinct (orthogonal) without losing the network's "magic"—its ability to handle unseen data.

Methodologies for Prevention

The paper outlines three distinct approaches to mitigate catastrophic forgetting:

  • Activation Sharpening & Prototype Biasing (French): Sharpening forces a few nodes to be highly active while suppressing others, creating a more localized representation. Prototype Biasing employs a secondary network to learn a "prototype" for each category. New items are constrained to resemble these prototypes while being separated from previous patterns based on how different their target outputs are.
  • Pretraining & Windowed Learning (Hetherington): Pretraining primes the network's weights by training it on a large, similar dataset prior to the sequential task. Windowed Learning acts as a form of rehearsal, where the network continues to process a small "window" of recently learned items alongside new ones.
  • Input Orthogonalization (Lewandowsky): A highly effective data preprocessing technique that converts asymmetric binary inputs (0 or 1) into symmetric vectors (-1 or 1), naturally reducing overlap.

Evaluation and Datasets

Rather than using specific modern benchmarks, the studies relied on generic large datasets to test scalability and datasets of similar item types for pretraining. The primary evaluation metric across these studies was the "method of savings," which measures how much faster a network can relearn "erased" old data compared to learning it entirely from scratch.

Key Insights and Results

  • Overlap is the Enemy: Forgetting peaks when high representation overlap in the hidden layer coincides with high error signals from the output. Reducing this overlap directly mitigates forgetting.
  • The Scalability Trap: Fully "localizing" representations (making only one or two neurons fire) prevents interference but destroys network capacity, as the model quickly runs out of neurons. The optimal solution is "semi-distributed" representations—patterns that are spread out but remain distinct. These were proven to reduce interference in backpropagation networks without harming generalization.
  • Prior Knowledge as a Buffer: Pretraining acts as a structural constraint. By learning the general structure of the data first, the network naturally slots new information into more orthogonal spaces. The authors note this can "virtually eliminate" catastrophic interference.
  • Prototype Biasing Success: This method successfully generated representations that were both distributed and well-separated, leading to a significant decrease in forgetting.
  • Symmetric Inputs: Shifting from (0, 1) to (-1, 1) significantly reduced interference by centering the data and reducing bias in weight updates.
  • Generalization vs. Interference: Reducing interference by altering how neurons respond to inputs (changing the "response surface") breaks generalization. However, reducing interference by finding a more robust weight pattern preserves both memory and generalization.

Limitations and Future Directions

While the proposed methods show promise, they come with trade-offs. Techniques like activation sharpening struggle to scale with larger datasets because they restrict the number of possible unique representations. Furthermore, methods relying on "static" hidden units impair generalization, marking them as a dead-end for flexible AI systems.

Finally, although the title questions if interference can be predicted, the text predominantly focuses on prevention. The exact mechanisms for predicting precisely when a network will fail remain an open area for future research.

Summary updated

BibTeX
@inproceedings{french1993catastrophic,
  author = {Robert M. French},
  title = {{Catastrophic Interference in Connectionist Networks: Can It Be Predicted, Can It Be Prevented?}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {1993},
}
Figure