From Isolation to Cooperation: An Alternative View of a System of Experts

Stefan Schaal, Christopher G. AtkesonNeurIPS 1995

This 1995 paper by Stefan Schaal and Christopher Atkeson introduces a novel way to organize "Systems of Experts" for machine learning. While traditional approaches make experts compete for data, this paper argues that letting them learn in isolation—and only cooperate during prediction—leads to much more robust incremental learning.

TL;DR: The authors present Receptive Field Weighted Regression (RFWR), an incremental learning system where "local experts" learn independently without competing for data. By isolating experts during training and blending their predictions only at inference, the system avoids "catastrophic interference" and can automatically detect relevant features and adapt to changing data distributions.

The Problem: Catastrophic Interference

In supervised learning, especially for robotics, systems often need to learn incrementally—improving as they see more data without forgetting what they already know. A common approach is a "Mixture of Experts" (MoE), where different sub-models (experts) handle different parts of the input space.

However, traditional MoE models usually involve competitive learning. Experts compete to explain new data points. If the distribution of input data changes (e.g., a robot moves from one task to another), this competition can cause "catastrophic interference." The system reassigns experts to the new data, effectively wiping out the knowledge they had about the old data. The authors aim to create a system that can grow and adapt locally without breaking the global model.

Methods: Receptive Field Weighted Regression (RFWR)

To solve this, the authors introduce Receptive Field Weighted Regression (RFWR).

Architecture of Receptive Field Weighted Regression

The key technical components of this architecture include:

  • Isolated Learning: Unlike standard MoE, experts do not compete. Each expert is an independent entity that only cares about data within its "receptive field" (a local region of the input space).
  • Locally Linear Models: Each expert uses a simple linear model to approximate the function in its immediate neighborhood.
  • Gaussian Receptive Fields: Each expert has a bell-shaped activation profile. The shape and orientation of this "field" are defined by a distance metric matrix, which the system learns.
  • Second-Order Optimization: The linear parameters are updated using a Newton method (Recursive Least Squares), while the receptive field parameters are updated via second-order gradient descent. This ensures very fast convergence compared to standard backpropagation.
  • Penalty Terms: To prevent receptive fields from shrinking to zero (overfitting to a single point), the cost function includes a regularization term that penalizes high curvature.
  • Dynamic Allocation: The system starts with no experts. It creates a new expert whenever it encounters data that isn't "covered" by an existing one, and it prunes experts that overlap too much or have high errors.

Datasets and Key Insights

The paper evaluates the algorithm on three main benchmarks: a synthetic 2D function, noisy high-dimensional data, and a simulated 2-degree-of-freedom (DOF) robot arm learning inverse dynamics.

Synthetic 2D function and learned receptive fields

Testing on these datasets revealed several core insights about the RFWR approach:

  • Isolation is a Feature, Not a Bug: By preventing competition, the authors decouple the experts. Adding new knowledge doesn't require re-negotiating the boundaries of old knowledge, making the system inherently resistant to forgetting.
  • Receptive Fields as Curvature Detectors: The distance metric effectively learns the Hessian of the function. In areas where the function is flat, the receptive fields grow large; in areas of high curvature, they become small and elongated to track the "ridges" of the data.
  • Automatic Feature Selection: By making the "ridge regression" parameters adjustable via gradient descent, each expert can independently decide which input dimensions are irrelevant and "zero them out." This allows the model to handle high-dimensional inputs where only a few dimensions matter locally.

Key Results

  • Speed and Accuracy: On the synthetic 2D function, RFWR achieved a Mean Squared Error (MSE) of 0.0025 after just 30 epochs (1 minute of computation). For comparison, a standard 3-layer neural network required 10,000 epochs (4 hours) to reach a similar result.
  • Zero Interference: When trained on three non-overlapping "stripes" of data sequentially, the model showed almost no performance degradation on the first stripe after learning the third. The MSE only rose slightly from 0.0025 to 0.003.
  • Dimensionality Reduction: In the noisy test (the 2D function augmented with 8 dimensions of pure noise), the system successfully identified all 8 irrelevant dimensions, increasing their ridge parameters to effectively ignore them.
  • Robot Control: The system generated 39 experts to control the robot arm. It successfully learned to draw a figure-8 in one area, moved to a second area to learn a new pattern, and could still perform the first pattern perfectly afterward.

Limitations and Future Work

While highly effective for incremental learning, the authors note a few limitations. The use of Newton methods and second-order gradients provides speed in terms of iterations but is computationally expensive per update, especially as the number of experts grows. Additionally, while local feature detection helps, further research is needed to see how well the "isolated expert" approach scales to extremely high-dimensional spaces. The authors suggest investigating "delta-rule" (first-order) versions of the algorithm to reduce the computational overhead per data point in future iterations.

Summary updated

BibTeX
@inproceedings{schaal1995from,
  author = {Stefan Schaal and Christopher G. Atkeson},
  title = {{From Isolation to Cooperation: An Alternative View of a System of Experts}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {1995},
}
Figure