Introduction
In high-stakes domains such as finance, medicine, and insurance, machine learning models must be more than just accurate; they must be trustworthy and auditable. While state-of-the-art gradient-boosted trees (e.g., XGBoost) provide exceptional predictive performance on tabular data, they often operate as "black boxes," utilizing hundreds of features in complex, non-linear ways. Conversely, inherently interpretable models like Explainable Boosting Machines (EBMs) or logistic regression often sacrifice accuracy for simplicity.
This research introduces EAGGA (Evolutionary Algorithm for Grouping Genetic Algorithm), a framework designed to automate the search for models that balance this trade-off. By treating interpretability as a multi-objective optimization problem, EAGGA generates a Pareto front of models, allowing users to select the optimal balance between performance and transparency.
The EAGGA Framework
The framework quantifies interpretability through three global sparsity measures:
- Feature Sparsity ($NF$): The fraction of available features utilized by the model.
- Interaction Sparsity ($NI$): The fraction of possible feature interactions permitted.
- Non-monotone Sparsity ($NNM$): The fraction of features allowed to exhibit complex, non-monotone effects (e.g., ensuring that increasing income never negatively impacts a credit score).
Unlike standard hyperparameter optimization, which focuses on settings like learning rate, EAGGA evolves the group structure of features. It partitions features into groups that share interaction and monotonicity constraints. To enhance efficiency, the algorithm uses data-driven "detectors" (such as information gain and Spearman correlation) to intelligently initialize the population before evolving these structures toward the Pareto optimal set.
Experimental Evaluation
The framework was evaluated across 20 binary classification tasks from the OpenML CC-18 and AutoML benchmarks, ranging from 4 to 970 features. EAGGA, using XGBoost as its base learner, was compared against a suite of tuned competitors, including EBMs, Random Forests, and Elastic-Nets.
Key Results
The primary metric for evaluation was the Mean Dominated Hypervolume, which measures the quality of the Pareto front—essentially how well a method optimizes both accuracy (AUC) and interpretability simultaneously.
As illustrated in the results, EAGGA consistently found models that "dominated" the competition. In many instances, EAGGA identified models that matched the AUC of a fully-tuned "black box" XGBoost while utilizing significantly fewer features and interactions. Furthermore, a significant portion of models produced by standard EBMs and Random Forests were found to be Pareto-dominated by EAGGA's solutions, meaning EAGGA could provide a model that was simultaneously more accurate and more interpretable.
Conclusion
EAGGA shifts the paradigm of interpretable machine learning from manual trial-and-error to automated multi-objective discovery. By providing a "menu" of Pareto-optimal models, it empowers domain experts to make informed decisions about the cost of interpretability. Future work will explore extending this framework to deep neural network architectures and implementing self-adaptive mutation rates to further refine the evolutionary search process.
Summary updated