PIE: A Partially Interpretable Model with Black-Box Refinement

Tong Wang, Jingyi Yang, Yunyi Li, Boxiang WangOpenalex Unknown 2025 · DOI · OpenAlex

Abstract

The Partially Interpretable Estimators (PIE) framework implements a novel predictive model that jointly trains an interpretable component and a black-box model. This approach achieves high predictive performance while maintaining partial model transparency, addressing the traditional trade-off between accuracy and interpretability in machine learning.

Introduction

In the field of machine learning, practitioners frequently encounter a conflict between model transparency and predictive power. Simple "white-box" models, such as linear regression, allow for clear attribution of feature contributions but often fail to capture the complex, non-linear relationships inherent in real-world data. In contrast, "black-box" models like XGBoost or deep neural networks offer state-of-the-art accuracy but provide little insight into the underlying decision-making process.

The PIE framework addresses this by creating a hybrid architecture. It aims to attribute as much of a prediction as possible to a transparent, additive component, utilizing a black-box model only to refine the remaining residuals. This ensures the model remains as understandable as a linear model while performing at the level of complex ensemble methods.

Methodology

The PIE approach relies on a two-part architecture trained through joint optimization:

  1. Interpretable Component: This consists of a linear additive model. To capture non-linearities without sacrificing transparency, numerical features are expanded using natural splines. The model employs Group Lasso (gglasso) regularization to perform feature selection, forcing entire groups of spline coefficients to zero to ensure a sparse and readable "white-box" component.
  2. Black-Box Refinement: An XGBoost model is integrated to capture high-dimensional interactions and complex patterns that the linear component might miss.
  3. Joint Optimization: Unlike simple stacking, PIE balances these components during training using specific hyperparameters.
Key input parameters for the PIE fitting process, including regularization penalties lambda1 and lambda2.

As shown in the parameter configuration, lambda1 controls the group lasso penalty for the interpretable part, while lambda2 determines the contribution of the black-box XGBoost model. The resulting model object contains both the interpretable coefficients and the ensemble of trees.

The output of the PIE_fit function, containing both the interpretable Betas and the black-box Trees.

Implementation and Data

The PIE package includes a robust preprocessing utility, data_process, which automates k-fold cross-validation setup, normalization, and spline expansion. For demonstration, the documentation highlights the Wine Quality Data benchmark, consisting of 5,197 samples of Portuguese "Vinho Verde" wine. The goal is to predict wine quality based on 11 numerical physicochemical tests and one categorical variable. This dataset effectively showcases PIE's ability to handle mixed feature types and spline expansions simultaneously.

Key Insights and Results

  • Additive Transparency: By utilizing splines within an additive framework, PIE allows for "Partial Model Transparency." Users can visualize the contribution of individual features across their ranges.
  • Residual Refinement: The black-box layer acts strictly as a correction. If a relationship is simple enough to be explained linearly, the regularization favors the interpretable path.
  • Performance: PIE achieves predictive accuracy comparable to pure black-box models on tabular datasets while maintaining an "understandability" score close to standard linear models.
  • Sparsity: The framework effectively reduces the number of active features in the white-box portion, making the interpretable component highly concise.

Conclusion

PIE offers a compelling solution for domains where both high accuracy and human-centric validation are required. By prioritizing transparency and using black-box methods only for refinement, it provides a model that aligns with expert reasoning without sacrificing performance. Future work may involve optimizing the computational overhead of joint training and exploring automated hyperparameter tuning for the balance between lambda1 and lambda2.

Summary updated

Abstract

Implements a novel predictive model, Partially Interpretable Estimators (PIE), which jointly trains an interpretable model and a black-box model to achieve high predictive performance as well as partial model. See the paper, Wang, Yang, Li, and Wang (2021) &lt;<a href="https://doi.org/10.48550%2FarXiv.2105.02410" target="_top">doi:10.48550/arXiv.2105.02410</a>&gt;.
BibTeX
@inproceedings{wang2025pie,
  author = {Tong Wang and Jingyi Yang and Yunyi Li and Boxiang Wang},
  title = {{PIE: A Partially Interpretable Model with Black-Box Refinement}},
  booktitle = {Openalex Unknown},
  year = {2025},
  doi = {10.32614/cran.package.pie},
}
Figure