On model selection and the disability of neural networks to decompose tasks

Marc Toussaint

Author: Marc Toussaint (2002)

Introduction

Standard feed-forward neural networks (FFNNs) are mathematically predisposed to "catastrophic forgetting," a phenomenon where learning a new task destroys previously acquired knowledge. While modern deep learning has scaled significantly since this research was presented at the International Joint Conference on Neural Networks (IJCNN), the fundamental geometric problems identified remain central to the challenges of continual learning and modular AI. This paper argues that traditional model selection focuses too much on complexity (what a network can represent) and ignores parametrization (how a network moves through function space during learning).

The Functional Metric and Decomposition

The core innovation of this work is the introduction of a framework based on differential geometry to analyze a network's "functional metric" ($g^{ab}$). This metric measures how a change in the network's parameters affects the output at different points in the input space.

The author proves a critical Decomposition Condition: for a network to truly decompose tasks—meaning learning one task does not interfere with another—this functional metric must be a block-diagonal matrix. If the metric is block-diagonal, the "ripples" from learning Task A are mathematically confined and cannot reach the territory of Task B. In standard FFNNs, however, weights are shared globally, and the cross-talk between different inputs is generically strong.

Experimental Comparison

To isolate these geometric behaviors, the paper utilizes a controlled regression problem involving two 3-bit patterns ($110 \rightarrow +1$ and $010 \rightarrow -1$). The networks are trained using an online sequential regime, alternating between the two patterns to trigger potential forgetting.

Key Results

In a standard FFNN, the network fails to maintain knowledge of the first task while learning the second. As shown in Figure 1, the error on the "untrained" pattern spikes every time the "trained" pattern is updated. The functional metric reveals high correlation ($g^{01}$) between the two tasks, indicating that the architecture is "wired" to forget because its functional metrics are rarely sparse.

Task interference in a standard feed-forward neural network.

Conversely, the paper introduces a "Softmax model" where hidden neurons compete for activation. This competition naturally pushes the network toward the block-diagonal ideal. As demonstrated in Figure 2, the Softmax model learns the second pattern with almost zero impact on the first. Its functional metric ($g^{01}$) drops toward zero, indicating successful task decomposition.

Task decomposition in a Softmax model.

Discussion and Implications

The research suggests that parametrization is destiny. It is not enough to know that a network can represent a function; the architecture dictates how it fills in the gaps between data points. By forcing neurons to compete, the network effectively specializes different parts of its architecture for different inputs, leading to "adaptation decomposition."

By testing a million random initializations, the author confirmed these are not fluke results: standard FFNNs almost always exhibit high task interference, while Softmax models have a high probability of achieving near-zero interference.

Limitations and Future Work

While the geometric insights are profound, the experiments are limited to small, 3-bit toy examples. It remains to be seen how these properties hold up in high-dimensional spaces like modern computer vision. Additionally, the author proposes a mathematical penalty term to favor architectures with low task interference during model selection, but the implementation of this penalty in a practical training loop is left for future research. The paper concludes by suggesting that evolutionary model selection may be the most viable path toward discovering truly modular architectures.

Summary updated

Abstract

A neural network with fixed topology can be regarded as a parametrization of functions, which decides on the correlations between functional variations when parameters are adapted. We propose an analysis, based on the differential geometry, that allows one to calculate these correlations. In practise, this describes how one response is unlearned while another is trained. Concerning conventional feed-forward neural networks we find that they generically introduce strong correlations, are predisposed to forgetting and inappropriate for task decomposition. Perspectives to solve these problems are discussed.
BibTeX
@misc{toussaint2002model,
  author = {Marc Toussaint},
  title = {{On model selection and the disability of neural networks to decompose tasks}},
  year = {2002},
  eprint = {nlin/0202038},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/nlin/0202038},
}
Figure