Abstract
This paper presents a framework for approximating the Rashomon set—the collection of all near-optimal models—for sparse Generalized Additive Models (GAMs). By representing this set as a searchable ellipsoid, the authors enable domain experts to interactively explore model space, assess variable importance ranges, and enforce constraints without sacrificing predictive performance. The proposed methods offer significant speedups and higher fidelity compared to traditional baselines, facilitating better human-AI collaboration in high-stakes decision-making.
Introduction
In traditional machine learning, algorithms typically identify a single "optimal" model based on a loss function. However, in high-stakes fields like medicine or criminal justice, a single model may not capture the nuanced requirements of domain experts. If the provided model violates unarticulated constraints (e.g., a specific variable's effect should be monotonic), experts often face an "interaction bottleneck," manually tweaking parameters through trial and error.
This research addresses this bottleneck by focusing on the Rashomon set: the set of all models within a function class that perform nearly as well as the best one. For Sparse Generalized Additive Models (GAMs)—interpretable models that represent targets as sums of univariate "shape functions"—finding this set is computationally challenging. This work introduces efficient methods to approximate and interact with the Rashomon set for sparse GAMs in classification contexts.
Methodology
The authors propose two primary methods to approximate the Rashomon set:
- Maximum Volume Inscribed Ellipsoid: For a fixed set of features (support set), the Rashomon set of a GAM is convex. The authors use gradient-based optimization to find the largest ellipsoid that fits inside this set, serving as a continuous, searchable proxy for all high-performing models.
- The Blocking Method: To avoid the computational explosion of exploring every feature combination, the authors demonstrate that an ellipsoid for a large support set can analytically yield Rashomon sets for any of its subsets by treating them as intersections with hyperplanes.
These tools allow for calculating Variable Importance Ranges, ensuring Constraint Satisfaction via quadratic programming, and Model Projection to return manually edited models back into the "good" range.
Key Insights: Beyond Point Estimates
- The Myth of the "Best" Model: Because many models perform similarly well, the choice between them should be driven by domain expertise rather than just a loss function.
- Variable Importance as a Range: A feature might appear vital in one model but be replaceable in another equally good model. Identifying variables with a high minimum importance across the Rashomon set helps distinguish truly essential features.
- Distinguishing Signal from Noise: Sudden "jumps" in shape functions can be evaluated across the entire Rashomon set to determine if they are persistent signals or random artifacts of a specific fit.
Key Results
The researchers evaluated their methods on four datasets: COMPAS (recidivism), FICO (credit risk), Diabetes (clinical), and MIMIC-II (ICU mortality).
- High Fidelity: The ellipsoid approximation significantly outperformed Hessian-based or bootstrap-based baselines in precision, ensuring that models within the ellipsoid were almost always truly "good" models.
- Massive Speedup: The blocking method (Method 2) is orders of magnitude faster than re-running optimization, finding new Rashomon sets in less than 0.001 seconds, compared to hundreds of seconds for Method 1.
Practical Utility and Conclusion
The authors demonstrated practical utility by taking a "jumpy" shape function from the MIMIC-II dataset and interactively smoothing it without sacrificing accuracy. While the framework makes the Rashomon set searchable, future work remains in human-computer interaction (HCI) to design interfaces that allow users to intuitively navigate these high-dimensional spaces. By moving beyond the single-model paradigm, this work provides a path toward more robust and expert-aligned machine learning systems.
Summary updated