Learning and Representation Change

Jeffrey C. SchlimmerAAAI 1987

Introduction

In the landscape of 1980s machine learning, systems were often constrained by their initial representational language. Jeffrey Schlimmer's 1987 work on the STAGGER system introduced a landmark approach to "Representation Change." Instead of being limited by predefined features, STAGGER was designed to dynamically rewrite its own descriptive language, allowing it to adapt to complex environments where initial categories might be insufficient.

Overcoming Representational Bias

A fundamental hurdle in early AI was representational bias. If a model's language is too restrictive, it cannot capture complex concepts; if it is too broad, the search space becomes computationally intractable. Most contemporary systems were static, unable to learn a "medium" category if they were only initialized with "small" and "large." STAGGER addressed this by autonomously modifying its symbolic categories based on continuous numeric input.

The STAGGER Architecture

STAGGER is a hybrid system composed of three interacting learning modules that form a feedback loop. This structure allows the system to refine its "alphabet" of features while simultaneously building complex "sentences" (concepts).

The interaction between STAGGER's three learning components.
  1. Weight Adjustment: Borrowing from connectionist principles and the Prospector expert system, STAGGER assigns weights to nodes using Logical Sufficiency (LS) and Logical Necessity (LN) measures. These weights are updated using Bayesian-style odds to reflect the predictive importance of specific features.
  2. Boolean Combination: To handle symbolic logic, the system employs a beam search triggered by prediction errors. It generates new features by combining existing ones using AND (to refine overly general concepts) or OR (to broaden overly specific ones).
  3. Numeric Partitioning: The system's most novel contribution is its ability to discretize continuous attributes. By tracking statistics for every observed value, STAGGER identifies "pivotal values" that maximize a utility function, effectively carving continuous ranges into useful symbolic intervals.

Dynamic Discretization through Utility

The numeric partitioner does not rely on arbitrary bins. Instead, it is class-driven, creating new categories only when they improve concept prediction. As shown in the utility graph below, the system identifies local maxima that distinguish positive examples from negative ones.

Utility function peaks identifying pivotal values for the 'size' attribute.

In a synthetic object-recognition task where the target concept was "red objects with a size between 5 and 15," STAGGER consistently discovered the correct numeric splits (e.g., $5 < \text{size} < 15$) within approximately 50 training examples.

Conclusion and Future Directions

STAGGER demonstrated that representational learning can be framed as a search through the space of possible languages. By marrying "soft" probabilistic weighting with "hard" symbolic logic and class-driven numeric partitioning, the system achieved efficient, real-time adaptation. While it faced challenges with non-linearly separable concepts like XOR, it laid the groundwork for modern autonomous representation change by showing how higher-level symbolic progress can emerge from lower-level numeric data.

Summary updated

BibTeX
@inproceedings{schlimmer1987learning,
  author = {Jeffrey C. Schlimmer},
  title = {{Learning and Representation Change}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {1987},
}
Figure