Introduction
In the landscape of 1980s machine learning, systems were often constrained by their initial representational language. Jeffrey Schlimmer's 1987 work on the STAGGER system introduced a landmark approach to "Representation Change." Instead of being limited by predefined features, STAGGER was designed to dynamically rewrite its own descriptive language, allowing it to adapt to complex environments where initial categories might be insufficient.
Overcoming Representational Bias
A fundamental hurdle in early AI was representational bias. If a model's language is too restrictive, it cannot capture complex concepts; if it is too broad, the search space becomes computationally intractable. Most contemporary systems were static, unable to learn a "medium" category if they were only initialized with "small" and "large." STAGGER addressed this by autonomously modifying its symbolic categories based on continuous numeric input.
The STAGGER Architecture
STAGGER is a hybrid system composed of three interacting learning modules that form a feedback loop. This structure allows the system to refine its "alphabet" of features while simultaneously building complex "sentences" (concepts).
- Weight Adjustment: Borrowing from connectionist principles and the Prospector expert system, STAGGER assigns weights to nodes using Logical Sufficiency (LS) and Logical Necessity (LN) measures. These weights are updated using Bayesian-style odds to reflect the predictive importance of specific features.
- Boolean Combination: To handle symbolic logic, the system employs a beam search triggered by prediction errors. It generates new features by combining existing ones using
AND(to refine overly general concepts) orOR(to broaden overly specific ones). - Numeric Partitioning: The system's most novel contribution is its ability to discretize continuous attributes. By tracking statistics for every observed value, STAGGER identifies "pivotal values" that maximize a utility function, effectively carving continuous ranges into useful symbolic intervals.
Dynamic Discretization through Utility
The numeric partitioner does not rely on arbitrary bins. Instead, it is class-driven, creating new categories only when they improve concept prediction. As shown in the utility graph below, the system identifies local maxima that distinguish positive examples from negative ones.
In a synthetic object-recognition task where the target concept was "red objects with a size between 5 and 15," STAGGER consistently discovered the correct numeric splits (e.g., $5 < \text{size} < 15$) within approximately 50 training examples.
Conclusion and Future Directions
STAGGER demonstrated that representational learning can be framed as a search through the space of possible languages. By marrying "soft" probabilistic weighting with "hard" symbolic logic and class-driven numeric partitioning, the system achieved efficient, real-time adaptation. While it faced challenges with non-linearly separable concepts like XOR, it laid the groundwork for modern autonomous representation change by showing how higher-level symbolic progress can emerge from lower-level numeric data.
Summary updated