ITL-LIME: Instance-Based Transfer Learning for Enhancing Local Explanations in Low-Resource Data Settings

Rehan Raza, Guanjin Wang, Kok Wai Wong, Hamid Laga, Marco FisichellaarXiv 2025 · arXiv

TL;DR

ITL-LIME is a novel explainable AI framework designed to provide stable and high-fidelity local explanations for black-box models in data-scarce environments. By replacing LIME's erratic synthetic perturbations with real instances transferred from a data-rich source domain and weighting them via self-supervised contrastive learning, it eliminates explanation instability and keeps local neighborhoods firmly on the true data manifold.

What Problem Does This Paper Address?

Local Interpretable Model-agnostic Explanations (LIME) is a cornerstone of Explainable AI (XAI), helping users understand complex "black-box" models by training a simple, interpretable surrogate model (like a linear regression) around a specific data point. To do this, LIME randomly perturbs the features of the target instance to generate a synthetic local neighborhood.

However, this approach severely breaks down in low-resource data settings (e.g., rare medical conditions or strict privacy domains). With limited data, LIME's random sampling ignores feature correlations, creating unrealistic synthetic instances that drift off the true data manifold. This leads to highly unstable explanations (giving different feature importances across multiple runs) and poor local fidelity, making the explanations untrustworthy for high-stakes decision-making.

Methods

The authors introduce ITL-LIME (Instance-Based Transfer Learning LIME), which completely bypasses synthetic data generation by leveraging real data from a related, well-sampled source domain.

The framework operates in three main steps:

  1. Source Domain Clustering: The data-rich source domain is partitioned using $K$-medoids clustering. Unlike $K$-means, $K$-medoids selects actual data points as cluster centers (prototypes), preserving real-world interpretability.
  2. Instance Transfer & Neighborhood Unification: For a given target instance, the algorithm finds the closest, label-consistent source prototype and extracts its cluster members. These real source instances are combined with the target instance's actual $k$-nearest neighbors from the target domain, forming a unified, realistic local neighborhood.
  3. Self-Supervised Contrastive Weighting: To refine this neighborhood, the authors employ SCARF (a contrastive learning method for tabular data). An encoder is trained on the unified neighborhood to learn robust semantic embeddings. Proximity weights are then assigned based on distances within this latent space rather than the raw feature space, mitigating the impact of noise and domain shifts.

Finally, a standard linear surrogate model is trained on this weighted, fully real dataset to generate the final explanation.

Figure 1: Workflow of the proposed ITL-LIME method.

Datasets

The framework was evaluated on two real-world healthcare benchmarks split into distinct source (large) and target (small) domains based on geographic/demographic shifts:

  • Diabetes Dataset (EHR 2015-2020):
  • Source: Mainland U.S. (DSD: 96,738 records)
  • Target 1: Virgin Islands (DTD1: 763 records)
  • Target 2: Guam (DTD2: 523 records)
  • Student Depression Dataset:
  • Source: Western India (SSD: 11,440 records)
  • Target 1: East India (STD1: 650 records)
  • Target 2: South India (STD2: 625 records)

Key Insights and Ideas

  • Ditching the Synthetic Myth: In low-data regimes, generating fake data points does more harm than good. Grounding the local neighborhood entirely in real instances (even from a slightly different domain) ensures the surrogate model learns valid decision boundaries.
  • Latent Space Proximity Matters: Measuring local proximity using raw Euclidean distance fails on noisy, correlated tabular data. Passing instances through a contrastive learning encoder (SCARF) aligns features semantically across domains before calculating weights.
  • Deterministic Stability: Because ITL-LIME relies on structured clustering and nearest-neighbor retrieval rather than random Gaussian perturbations, it effectively cures LIME's notorious randomness problem.

Key Results

The authors compared ITL-LIME against standard LIME and top variants (Bay-LIME, S-LIME, US-LIME, A-LIME, D-LIME) using Support Vector Machines (G-LIBSVM) and Deep Neural Networks (DNN):

  • Fidelity: ITL-LIME consistently achieved the highest F1-scores and AUCs. For instance, on the DTD1 dataset with a G-LIBSVM black-box, ITL-LIME outperformed standard LIME by 8.6 percentage points in F1-score (0.6917 vs. 0.6061) and 9.6 points in AUC (0.9029 vs. 0.8069).
  • Stability: Measured via the Jaccard Coefficient across multiple runs, ITL-LIME achieved a perfect 100% stability score (1.0000) across all datasets and models, whereas standard LIME dropped as low as 0.75.
  • Robustness: ITL-LIME achieved the lowest Local Lipschitz Estimator (LLE) values across the board, proving it is significantly less sensitive to small input noise than its competitors.
Table 1: The average testing fidelity results for all explanation methods on 40 randomly selected test instances from the target domain. Higher values (↑) indicate better fidelity.

Limitations and Future Work

While ITL-LIME marks a massive step forward for low-resource XAI, the authors note a few areas for development:

  • Broader Evaluation: The method currently needs validation across a wider variety of diverse domain datasets beyond the two healthcare-related benchmarks tested.
  • Computational Overhead: Training a contrastive learning encoder and running $K$-medoids clustering on a large source dataset adds computational complexity. The authors plan to conduct a formal analysis of runtime overhead and practical feasibility in future work.

Summary updated

Abstract

Explainable Artificial Intelligence (XAI) methods, such as Local Interpretable Model-Agnostic Explanations (LIME), have advanced the interpretability of black-box machine learning models by approximating their behavior locally using interpretable surrogate models. However, LIME's inherent randomness in perturbation and sampling can lead to locality and instability issues, especially in scenarios with limited training data. In such cases, data scarcity can result in the generation of unrealistic variations and samples that deviate from the true data manifold. Consequently, the surrogate model may fail to accurately approximate the complex decision boundary of the original model. To address these challenges, we propose a novel Instance-based Transfer Learning LIME framework (ITL-LIME) that enhances explanation fidelity and stability in data-constrained environments. ITL-LIME introduces instance transfer learning into the LIME framework by leveraging relevant real instances from a related source domain to aid the explanation process in the target domain. Specifically, we employ clustering to partition the source domain into clusters with representative prototypes. Instead of generating random perturbations, our method retrieves pertinent real source instances from the source cluster whose prototype is most similar to the target instance. These are then combined with the target instance's neighboring real instances. To define a compact locality, we further construct a contrastive learning-based encoder as a weighting mechanism to assign weights to the instances from the combined set based on their proximity to the target instance. Finally, these weighted source and target instances are used to train the surrogate model for explanation purposes.
BibTeX
@misc{raza2025itl,
  author = {Rehan Raza and Guanjin Wang and Kok Wai Wong and Hamid Laga and Marco Fisichella},
  title = {{ITL-LIME: Instance-Based Transfer Learning for Enhancing Local Explanations in Low-Resource Data Settings}},
  year = {2025},
  eprint = {2508.13672},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2508.13672},
}
Figure