Abstract
Many researchers have noted the importance of combining inductive and analytical learning, yet we still lack combined learning methods that are effective in practice. We present here a learning method that combines explanation-based learning from a previously learned approximate domain theory, together with inductive learning from observations. This method, called explanation-based neural network learning (EBNN), is based on a neural network representation of domain knowledge. Explanations are constructed by chaining together inferences from multiple neural networks. In contrast with symbolic approaches to explanation-based learning which extract weakest preconditions from the explanation, EBNN extracts the derivatives of the target concept with respect to the training example features. These derivatives summarize the dependencies within the explanation, and are used to bias the inductive learning of the target concept. Experimental results on a simulated robot control task show that EBNN requires significantly fewer training examples than standard inductive learning. Furthermore, the method is shown to be robust to errors in the domain theory, operating effectively over a broad spectrum from very strong to very weak domain theories.
Introduction
In their seminal 1993 paper, Sebastian Thrun and Tom Mitchell introduced Explanation-Based Neural Network learning (EBNN). This approach represents a significant milestone in machine learning, specifically addressing the trade-off between "data-hungry" inductive methods and "knowledge-hungry" analytical methods. EBNN leverages the strengths of neural networks to provide a robust framework that utilizes prior knowledge to accelerate learning without sacrificing the ability to learn from raw data when that knowledge is flawed.
The Challenge: Bridging Induction and Analysis
Historically, machine learning research was bifurcated into two distinct paradigms. Inductive learning (e.g., standard backpropagation) relies solely on observations, requiring vast datasets to achieve generalization. Conversely, Analytical or Explanation-Based Learning (EBL) uses a "domain theory" to explain individual examples, enabling learning from a single instance but suffering from extreme brittleness if the domain theory is even slightly inaccurate.
Thrun and Mitchell proposed a unified method designed to occupy the middle ground of this spectrum. Their goal was a system that could utilize high-quality prior knowledge to speed up learning while "degrading gracefully" to standard inductive performance as the quality of that knowledge diminishes.
The EBNN Approach: Learning from Explanations
The core of EBNN is the representation of domain knowledge as a collection of neural networks. For instance, in a robot control task, individual networks might represent the expected outcomes of specific actions. The learning process follows a three-step cycle for every observed episode:
- Explain: The system uses its current domain theory to predict the outcome of an observed episode. Since these theories are often learned and imperfect, predictions will naturally deviate from actual observations.
- Analyze: Unlike symbolic EBL, which extracts logical rules, EBNN treats neural networks as differentiable functions. Using the chain rule, it calculates the partial derivatives (slopes) of the goal state with respect to the input features. These slopes serve as the "explanation," highlighting which features were most influential.
- Refine: The target function (such as a Q-function) is updated using a dual objective: an inductive component that fits the actual observed values, and an analytical component that fits the extracted slopes. This biases the network's "shape" to match the sensitivities suggested by the domain theory.
To manage imperfect knowledge, the authors introduced the LOB* heuristic. This mechanism weights the analytical component based on the accuracy of the domain theory's prediction. If prediction error is high, the system discounts the slopes and relies more heavily on the inductive data.
Core Innovations
- Slopes as Bias: Using derivatives as the medium for explanation is a profound shift from rigid logical rules. In continuous domains, knowing the sensitivity of an outcome to specific features provides a smooth, effective bias for generalization.
- The LOB* Heuristic: This heuristic addresses the "brittleness" problem of traditional EBL. By monitoring its own prediction errors, the agent gains a form of "self-awareness" regarding the reliability of its internal models.
- Learning to Learn: Because the domain theory is itself composed of neural networks, it can be refined inductively over time. As the agent explores, its domain theory improves, which subsequently enhances the effectiveness of its analytical learning for future tasks.
Experimental Evaluation: Robot Navigation
The authors tested EBNN in a simulated robot navigation environment where the agent had to reach a goal while avoiding obstacles. To evaluate robustness, they generated nine different domain theories by pre-training action networks on varying amounts of random data, simulating a spectrum from "clueless" to "expert" knowledge.
Results and Discussion
The experimental results demonstrated that EBNN reached high performance levels with significantly fewer training episodes than pure inductive Q-learning. Crucially, the system exhibited "graceful degradation": as domain theory accuracy decreased, EBNN's performance remained at or above the level of pure inductive learning, never falling below the "no-knowledge" baseline.
Without the LOB* weighting, poor domain theories were found to actually hinder the learner, confirming that this heuristic is essential for integrating learned, imperfect models. While EBNN is propositional and may struggle with first-order logical concepts, it remains a foundational approach for differentiable, knowledge-guided machine learning.
Summary updated