Abstract
Daniel Weld’s 1990 paper introduces a formal framework for automated model switching. By treating simpler models as mathematical limits of more complex ones (via "approximation reformulations"), the paper demonstrates that the difficult task of comparing two different models can be reduced to the much simpler task of analyzing parameter changes within a single model.
The Problem: Inter-Model Comparative Analysis
In the world of model-based reasoning, there is no "one size fits all" model. A molecular-level description of a car is useless for diagnosing a transmission problem, just as a high-level macro model can’t explain chemical fuel combustion. Engineers and AI systems must frequently switch between models of varying complexity to find the right balance between accuracy and computational cost.
The challenge is inter-model comparative analysis: if your current model’s predictions don't match reality, how do you know which alternative model will fix the discrepancy? Historically, comparing two different models (with different variables and equations) was computationally expensive and theoretically murky. Weld seeks a way to qualitatively predict how shifting from Model A to Model B will change the predicted behavior of a system.
Methods: Reformulations and Approximations
The paper introduces two core concepts to bridge the gap between models:
- Reformulation Functions: These are mathematical mappings that link the vocabulary of a complex model to a simpler one. For example, a model using polar coordinates can be "reformulated" into rectangular coordinates.
- Approximation Reformulations: Weld defines a simple model as an approximation of a complex one if the complex model contains a fitting parameter that, when pushed to a limit, makes the two models' behaviors identical.
Example: A model of a "frictionless" block is an approximation of a model with friction, where the "coefficient of friction" is the fitting parameter and its limit is zero.
The CA Reduction Theorem
The paper’s primary theoretical contribution is proving that if two models are related by an approximation reformulation, the problem of comparing them (inter-model) reduces to an intra-model problem. Instead of comparing two different sets of equations, you simply perform qualitative analysis on the fitting parameter within the more complex model.
Evaluation: Physical Systems and SAM
As a classic paper in qualitative reasoning, it evaluates the theory using physical system models and a software implementation:
- Physical System Models: Toy problems common in AI physics, such as a block sliding down an inclined plane (with and without friction) and a pulley system with weights connected by an elastic vs. inelastic rope.
- SAM (System for Automated Modeling): A Common Lisp implementation used to validate the theory on a "Graph of Models" (GoM).
The SAM program demonstrated that a system could automatically detect a discrepancy in a simulation (using Runge-Kutta integration), search a Graph of Models, and qualitatively determine which more complex model would likely resolve the error.
Key Insights and Results
- The Asymmetry of Models: Weld notes that you can usually map a "large" (complex) model onto a "small" (simple) one, but not necessarily the other way around. This reflects the intuitive idea that simplification loses information.
- Parameter Tuning as Model Shifting: Switching models isn't a discrete jump. By identifying fitting parameters (like spring constants or friction coefficients), we can treat a model shift as a continuous movement along a parameter scale.
- Generality: In an audit of the simple mechanics domain, Weld found 17 common simplifying assumptions. Remarkably, 16 out of 17 (e.g., "massless string," "frictionless surface") could be formally expressed as approximation reformulations.
Limitations and Future Work
The theory currently only works for "approximation reformulations." While these cover many common engineering assumptions, they don't cover every type of model shift (e.g., shifting from a particle view to a wave view in physics). Furthermore, the SAM implementation was limited to simple projection functions and struggled with systems that have multiple "operating regions." Weld suggests combining this approach with compositional modeling to allow for more granular, "detail-on-demand" modeling.
Summary updated