Introduction
In scientific domains such as genomics and healthcare, identifying how variables interact is often more critical than understanding their individual effects. For instance, a drug's efficacy might depend significantly on a patient's specific genetic markers. While neural networks excel at capturing these complex relationships, they are frequently criticized as "black boxes." Existing interpretation methods often struggle with locality (finding point-specific interactions), rigidity (requiring pre-specified functional forms), and a lack of statistical uncertainty, which can lead to high Type I error rates.
This paper introduces a robust framework for detecting global pairwise interactions using Bayesian Neural Networks (BNNs). The core contribution is the Bayesian Group Expected Hessian (GEH), a metric that identifies non-linear relationships across a dataset while providing principled measures of uncertainty.
Methodology
The proposed approach consists of two primary components: hybrid modeling and the GEH metric.
Hybrid Modeling
To focus the network's capacity on complex interactions, the authors employ a hybrid architecture that separates linear "main effects" from non-linear "interactions" modeled by the BNN. Uncertainty is captured using Concrete Dropout, ensuring that the model's confidence intervals are well-calibrated.
Bayesian Group Expected Hessian (GEH)
The Hessian—the second derivative of the output with respect to two inputs—is a standard mathematical tool for measuring interaction. However, simple aggregation methods like the Absolute Expected Hessian (AEH) or Expected Absolute Hessian (EAH) often fail due to signal cancellation or noise sensitivity.
The GEH addresses this by partitioning the data into $M$ clusters. It calculates the average interaction within each group before aggregating them. This grouping mechanism allows for a tunable trade-off: a small $M$ effectively cancels noise, while a larger $M$ captures finer-grained signals. Because the model is Bayesian, the GEH yields a distribution rather than a point estimate, allowing for the calculation of 95% credible intervals to determine statistical significance.
Experimental Evaluation
The framework was evaluated using synthetic and real-world datasets:
- Simulated Data: A synthetic dataset with 8 features and 7 interaction types (e.g., multiplicative, exponential, logarithmic) to test ground-truth recovery.
- Real-world Regression: Included the California Housing dataset (predicting prices), Bike Sharing (rental counts), and Energy Efficiency (building loads).
- MNIST: Demonstrated interactions between high-level features in image classification.
Key Results
The GEH method consistently outperformed standard baselines, including Lasso, Neural Interaction Detection (NID), and SHAP.
Quantitative Performance
In simulations, GEH ranked true interactions higher than competing methods across various functional forms. Its performance closely tracked that of an "Oracle" model with prior knowledge of the true functional forms, even as the signal-to-noise ratio varied.
Real-World Interpretability
On the California Housing dataset, the model successfully identified the strong joint interaction between latitude and longitude, confirming that location is a coupled effect. Similar intuitive interactions were recovered in the Bike Sharing and Energy Efficiency datasets.
Discussion and Limitations
The authors provide theoretical guarantees showing that as BNN prediction accuracy improves, the error in detected interactions diminishes. However, some limitations remain. The method's reliance on K-means clustering may be suboptimal for data on complex manifolds. Additionally, calculating Hessians across multiple Monte Carlo samples is computationally intensive for high-dimensional feature spaces. Finally, the "representation learning" benefits of this approach are most apparent on larger datasets; on very small samples, simpler linear models may still be competitive.
Conclusion
The Bayesian Group Expected Hessian offers a principled, uncertainty-aware framework for discovering global feature interactions. By bridging the gap between local sensitivity and global aggregation, it provides a powerful tool for scientific discovery in complex, non-linear domains.
Summary updated