Learning to Learn Decision Trees

Vlad G. Dabija, Katsuhiko Tsujino, Shogo NishidaAAAI 1992

Abstract

This paper presents Meta-KAISER, a recursive learning apprentice system designed to bridge the gap between mathematically optimal decision trees and those deemed "proper" by human experts. By treating the domain of decision tree construction as a field of study itself, Meta-KAISER observes expert corrections to flawed models to build a meta-theory of improprieties. This approach allows the system to learn subtle domain-specific heuristics, reducing cognitive load for experts and improving the quality of generated expert systems in fields such as electrical fault diagnosis.

Introduction

In the early 1990s, inductive learning algorithms like ID3 were widely used to generate decision trees from data. However, a significant practical hurdle remained: the trees produced were often "improper" according to domain specialists. While a tree might achieve high statistical accuracy, it could contain redundant nodes, violate domain constraints, or follow a logic that felt "unnatural" to an expert.

The fundamental challenge is that there is no universal, formal definition of a "good" decision tree. Expert preferences are often rooted in "aesthetic feelings" and deep-seated domain experience that are difficult to formalize manually. This research introduces a meta-learning framework that learns these heuristics directly from expert interactions.

The KAISER and Meta-KAISER Architecture

The proposed system utilizes a two-tier recursive architecture to manage knowledge acquisition and refinement.

The system structure of KAISER, showing the flow from examples and domain knowledge to impropriety detection and generation of explanations/questions.

KAISER: The Base System

KAISER (Knowledge Acquisition Inductive System driven by Explanatory Reasoning) serves as the primary interface. It constructs decision trees from raw data and domain rules. Its core function is "Impropriety Detection," where it identifies potential flaws—such as nodes with insufficient support—and prompts the expert for corrective actions.

Meta-KAISER: The Learning Apprentice

Meta-KAISER acts as a "Learning Apprentice" that studies the "domain of decision trees." Every time an expert disagrees with a suggestion or manually adjusts a tree, Meta-KAISER captures this interaction as a training example.

Key features of this architecture include:

  • Recursive Logic: Meta-KAISER uses decision trees to represent its meta-knowledge about how to fix other decision trees.
  • Bootstrapping: The system begins with a basic, hand-coded theory of improprieties and refines it through expert feedback.
  • Heuristic Learning: It focuses on Heuretics—the study of heuristics—to understand not just what is wrong, but the underlying reason and the optimal path to correction.

Experimental Evaluation

The system was evaluated through the development of real-world expert systems at Mitsubishi Electric Corp, including:

  • Oil-isolated electric transformer diagnosis: Classifying faults (e.g., arcing, overheating) based on gas composition (H2, CH4, C2H6).
  • Electric motor failure diagnosis.
  • Tap changer maintenance for transformers.

Unlike static benchmarks, the evaluation focused on the live interaction history between the system and human experts, measuring how the system's suggestions aligned with expert intent over time.

Key Insights: Expert Discontent as Data

A central insight of this work is that expert disagreement is the most valuable signal. When an expert rejects a system suggestion, it provides a "negative example" that forces the meta-theory to evolve. This allows Meta-KAISER to learn domain-dependent aesthetics; for instance, the logical structure required for medical diagnosis may differ significantly from that of power plant maintenance.

By prioritizing qualitative heuristics over rigid mathematical formulas, the system can manage complex logical issues like "Twin-nodes" (redundant attribute references) and "Inseparable examples" (identical data points belonging to different classes).

Results and Convergence

Preliminary results indicated that Meta-KAISER successfully converged toward a "generally accepted basic theory" for specific domains. As the system was used, the frequency of Meta-KAISER being invoked by experts decreased, suggesting that the base system (KAISER) was increasingly providing the "right" suggestions on the first attempt.

An example of a complex decision tree structure generated or managed by the system, illustrating the depth of logical branching.

The complexity of the learned trees (as seen in Figure 3) demonstrates the system's ability to handle multi-layered decision logic that incorporates both data-driven induction and expert-refined heuristics.

Conclusion and Future Work

Meta-KAISER demonstrates that a "learning to learn" approach can effectively capture the subtle, aesthetic preferences of human experts. While the authors acknowledge that a universal theory of decision trees may be unattainable due to domain-specific requirements, the system provides a robust framework for reducing the cognitive load on experts during knowledge acquisition. Future work aims to isolate a "core" general theory of decision trees that can serve as a more advanced starting point for new domains.

Summary updated

BibTeX
@inproceedings{dabija1992learning,
  author = {Vlad G. Dabija and Katsuhiko Tsujino and Shogo Nishida},
  title = {{Learning to Learn Decision Trees}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {1992},
}

Loading related content...

Figure