SIFOTL: A Principled, Statistically-Informed Fidelity-Optimization Method for Tabular Learning

Shubham Mohole, Sainyam GalhotraarXiv 2025 · DOI · arXiv

Abstract

Identifying the drivers of distribution shifts in tabular data is critical for high-stakes domains like healthcare and public policy. However, analysts often face significant hurdles, including strict privacy regulations, pervasive observational noise, and the "black box" nature of automated tools. This paper introduces SIFOTL (Statistically-Informed Fidelity-Optimization Method for Tabular Learning), a novel framework that identifies interpretable population segments responsible for data shifts. SIFOTL leverages a multi-stage pipeline involving LLM-guided feature engineering, a twin-model XGBoost approach to separate signal from noise, and a Pareto-optimized decision tree for rule extraction. Experimental results on real-world and synthetic datasets demonstrate that SIFOTL significantly outperforms industry-standard baselines, maintaining high fidelity even in the presence of substantial noise.

Introduction

In fields such as healthcare, data distribution shifts—changes in underlying patterns over time or across locations—can indicate anything from the success of a new policy to a critical failure in a clinical model. Pinpointing the specific demographic or clinical segments driving these shifts is essential for actionable insights.

Current analysis methods face a "trilemma" of challenges:

  1. Privacy: HIPAA and similar regulations often restrict access to individual-level data, forcing reliance on summary statistics.
  2. Noise: Real-world datasets are frequently contaminated with system glitches or data entry errors that mimic meaningful shifts.
  3. Interpretability: Stakeholders require human-readable rules (e.g., "Patients over 65 with specific comorbidities") rather than opaque model scores.

Existing solutions, such as Google’s BigQuery Contribution Analysis (BQCA), often fail to adequately address noise or provide the necessary granularity for policy-level decision-making.

Methodology: The SIFOTL Framework

SIFOTL addresses these challenges through a principled, multi-stage pipeline designed to maximize both interpretability and statistical fidelity.

Comparison of traditional analysis methods versus the SIFOTL pipeline, highlighting the Pre-Processing, Twin XGBoost, and Segment Builder stages.

1. Privacy-Preserving Feature Engineering

To maintain privacy, SIFOTL does not expose raw data to Large Language Models (LLMs). Instead, it provides the LLM with privacy-safe statistical summaries (e.g., $\chi^2$ tests, correlations). The LLM then acts as a "metadata architect," suggesting and generating Python code for new features that bridge the gap between raw variables and domain-specific logic.

2. The Twin-Model Approach

Rather than attempting to "clean" the data beforehand, SIFOTL explicitly models noise. It trains two separate XGBoost models:

  • Model C (Signal): Predicts the likelihood that a record belongs to the "intervention" or real shift.
  • Model N (Noise): Predicts the likelihood that a record represents observational noise (e.g., billing system errors).

3. Pareto-Weighted Decision Tree

The "secret sauce" of SIFOTL is its use of a penalty parameter ($\alpha$) within a shallow decision tree. This parameter balances capturing the real signal (fidelity) against ignoring records flagged as noise. By identifying the "knee" of the Pareto front, the system automatically selects a robust, interpretable explanation for the shift.

Experimental Evaluation

The authors evaluated SIFOTL using two primary data sources:

  • MEPS (Medical Expenditure Panel Survey): A real-world dataset where a policy shift (e.g., a new Medicare drug subsidy) was simulated.
  • Synthea ABM: A synthetic Electronic Health Record (EHR) generator used to create 18 diverse "stress-test" scenarios with varying levels of injected noise (N1 and N2).

Quantitative Results

SIFOTL demonstrated superior performance across all tested scenarios, particularly in noisy environments where traditional methods failed.

Performance comparison (F1 Score) of SIFOTL against BQCA and standard statistical tests across different tasks (T1-T3) and noise levels (N0-N2).

On the MEPS dataset, SIFOTL achieved an F1 score of 0.85, significantly higher than BQCA (0.46) and standard statistical tests (0.20). As shown in the table above, SIFOTL maintains high F1 scores (0.75–0.96) even under heavy noise, whereas baseline performance often drops precipitously.

Conclusion and Future Work

SIFOTL represents a significant advancement in interpretable tabular learning. By treating noise as a first-class citizen and leveraging LLMs for statistical synthesis, the framework provides a path for AI-assisted analysis in privacy-sensitive environments. Future work will focus on automating LLM code correction to improve reliability and incorporating formal Differential Privacy guarantees to meet the gold standard for healthcare applications.

Summary updated

BibTeX
@misc{mohole2025sifotl,
  author = {Shubham Mohole and Sainyam Galhotra},
  title = {{SIFOTL: A Principled, Statistically-Informed Fidelity-Optimization Method for Tabular Learning}},
  year = {2025},
  eprint = {2507.17979},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2507.17979},
}
Figure