Abstract
The trade-off between model precision and interpretability remains a significant challenge in machine learning, particularly for high-stakes domains like climate science. This paper introduces a model-agnostic framework that employs multi-objective evolutionary algorithms to discover mathematical models represented as Directed Acyclic Graphs (DAGs). By simultaneously optimizing for accuracy, complexity, and robustness, the framework generates a Pareto frontier of potential models. We demonstrate the framework's efficacy across machine learning pipelines, algebraic expressions, and differential equations, using complex sea surface height data as a benchmark.
Introduction
In many scientific and engineering fields, the "interpretability problem" hinders the adoption of high-performance machine learning models. While complex "black box" models offer high accuracy, they often lack transparency, making it difficult to verify their underlying logic or ensure generalizability. Conversely, simple models are interpretable but frequently lack the necessary precision for complex tasks.
A recent study by Alexander Hvatov and colleagues at ITMO University proposes a "composite model discovery" approach. Instead of post-hoc explanation of complex models, they suggest evolving models that are inherently balanced between accuracy and simplicity from the outset.
The Model Discovery Framework
The core of the proposed approach is the representation of models as Directed Acyclic Graphs (DAGs) composed of "atomic" units—ranging from simple trigonometric functions to complex machine learning regressors. The discovery process is structured into three primary steps: defining the search space, selecting an optimization strategy, and evaluating performance.
Refined Evolutionary Operators
To navigate the vast search space of potential model structures, the authors utilize Genetic Programming (GP). Standard GP often suffers from "bloat," where models become excessively complex without corresponding gains in performance. To mitigate this, the framework introduces refined evolutionary operators:
- Customized Crossover and Mutation: These operators are designed to maintain structural validity when swapping or altering sub-graphs, ensuring that input/output dimensions remain consistent.
- Regularization via Pruning: A specialized operator assesses the "dispersion ratio" (e.g., $R^2$) at each node. Nodes that contribute insignificantly to the final output are pruned, keeping the model compact and interpretable.
Multi-Objective Optimization
Rather than seeking a single optimal solution, the algorithm employs the MOEA/DD (Multi-Objective Evolutionary Algorithm based on Decomposition and Dominance). This approach identifies a "Pareto frontier"—a set of models representing the best possible trade-offs between accuracy, complexity, and robustness. This allows domain experts to select a model that fits their specific requirements for transparency versus performance.
Experimental Validation
The framework was validated using sea surface height (SSH) time series data generated by the NEMO (Nucleus for European Modelling of the Ocean) simulation for the Arctic Ocean. This dataset, featuring hourly resolution over one year, provides a rigorous testbed due to its multi-scale variability.
Key Results
The framework demonstrated versatility across three distinct modeling paradigms:
- Machine Learning Pipelines: The system identified that while decision trees could achieve high accuracy, single ridge regressions were significantly more robust across different data partitions.
- Algebraic Expressions: The algorithm successfully reconstructed spectral-like decompositions of the SSH data. As complexity increased, the model added finer-grained terms without disrupting the core structure of simpler approximations.
- Differential Equations: The system evolved equations approximating the underlying physical processes. However, a notable limitation was observed: without spatial data, the algorithm could not discover spatial derivatives, highlighting the dependency on the informational content of the input data.
Conclusion
The proposed multi-objective evolutionary approach offers a robust path toward discovering interpretable mathematical models. By treating the trade-off curve itself as a form of explanation, the framework empowers researchers to understand the cost of complexity. Future work will focus on unifying algebraic and machine learning components into hybrid models that combine the flexibility of ML with the physical grounding of mathematical physics.
Summary updated