Learning Engineering Models with the Minimum Description Length Principle

R. Bharat Rao, Stephen C. Y. LuAAAI 1992

Introduction

In complex engineering domains, physical phenomena are rarely homogeneous. A single mathematical relationship that describes engine performance at low RPMs often fails at higher speeds. While standard machine learning tools like decision trees attempt to partition data, they often do so arbitrarily before fitting models. Conversely, "fit-and-split" systems discover equations first but struggle to define their boundaries. This paper introduces KEDS-MDL, a system that automatically discovers mathematical models for non-homogeneous engineering data by balancing model accuracy with simplicity using the Minimum Description Length (MDL) principle.

Methodology: KEDS and the MDL Principle

The authors combine the Knowledge-based Equation Discovery System (KEDS) with the MDL principle to create a robust modeling framework.

KEDS: Conceptual Clustering

Unlike traditional methods that split data into rigid boxes, KEDS samples data to find candidate polynomial equations (e.g., linear or quadratic). It then identifies the specific "region" in the predictor space where each equation remains valid.

The process of fitting equations to specific regions in the predictor space.

The MDL Principle as a Global Judge

The MDL principle serves as the system's objective function. It posits that the best model minimizes the total bits required to encode both the model itself (complexity penalty) and the errors it makes (error penalty). The KEDS-MDL algorithm iteratively selects region-equation pairs that reduce the global description length, effectively preventing overfitting by penalizing overly complex models that don't significantly improve accuracy.

Experimental Evaluation

The system was evaluated using OTTONET, an internal combustion engine simulator. The researchers used four predictor variables—compression ratio, inlet pressure, inlet temperature, and fuel/air equivalence ratio—to model three response variables: gross indicated thermal efficiency ($\eta_{gr}$), net indicated thermal efficiency ($\eta_{net}$), and volumetric efficiency ($\eta_{vol}$).

Key Results

KEDS-MDL demonstrated significant improvements in both accuracy and computational efficiency.

  • Superior Accuracy: The MDL-based selection metric consistently outperformed other heuristics, such as choosing based solely on local accuracy or region size. As shown in the table below, the MDL approach yielded the lowest error rates across all efficiency metrics.
  • Computational Efficiency: KEDS-MDL reached solutions significantly faster than breadth-first search methods. In one test, it achieved a solution in 1,411 seconds, compared to 4,701 seconds for the standard approach.
  • Robustness: Even when computational resources were restricted to 1/10th of the original allocation, the MDL-guided search remained highly competitive, proving its effectiveness as a search heuristic.
Comparison of error rates for different selection metrics across response variables.

Conclusion and Future Work

By treating the problem space as a collection of regions governed by distinct functions, KEDS-MDL naturally handles the "regime changes" inherent in physical systems. While currently limited by a predefined library of polynomial templates, the system provides a powerful foundation for automated model discovery. Future work aims to incorporate deeper domain knowledge and apply these techniques to "inverse engineering" in fields like VLSI design, where discovered models can guide the synthesis of new, high-performance components.

Summary updated

BibTeX
@inproceedings{rao1992learning,
  author = {R. Bharat Rao and Stephen C. Y. Lu},
  title = {{Learning Engineering Models with the Minimum Description Length Principle}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {1992},
}
Figure