Temporal Feature Attribution for Dynamic Models

141 concepts80 papers1 proposal

Imagine a machine learning model monitoring a patient in an intensive care unit. At hour 2, a spike in heart rate might be a critical warning sign of sepsis. By hour 48, after major surgery, that same elevated heart rate might be completely expected, while a sudden drop in blood pressure becomes the real danger.

If you ask a traditional Explainable AI (XAI) tool—like standard SHAP or LIME—to explain the model's risk prediction over the entire stay, it will compress the importance of heart rate and blood pressure into single, static scalar values. By averaging a feature's impact across the entire timeline, static methods wash out critical signals, or worse, completely miss "crossing effects" where a feature's impact reverses direction over time.

To solve this, a growing body of research focuses on temporal feature attribution: treating feature importance not as a fixed number, but as a dynamic function that evolves over time.

From Scalar Scores to Attribution Curves

The foundational shift in temporal attribution is redefining the output of an explanation. In domains like survival analysis or time-series forecasting, models often output continuous trajectories—such as a survival probability curve over time—rather than a single classification score.

To interpret these functional outputs, researchers extend classical cooperative game theory and gradient-based attribution to the temporal domain. For instance, Mateusz Krzyziński et al. introduced SurvSHAP(t): Time-dependent explanations of machine learning survival models, a framework that calculates Shapley values independently at every discrete time point. This produces a time-varying contribution curve for each variable, allowing users to see exactly when a risk factor transitions from protective to harmful.

To scale this intuition to complex deep learning architectures, Sophie Hanna Langbein et al. developed Gradient-based Explanations for Deep Learning Survival Models. By extending sensitivity and decomposition methods across the entire temporal domain, they compute feature attributions across continuous survival curves significantly faster than model-agnostic alternatives.

Shifting Baselines to Explain Risk Trajectories

Why did a system's risk score suddenly jump between hour 12 and hour 13? Standard attribution methods typically evaluate a model's prediction relative to a static baseline, such as an all-zero input vector or an average population profile. While this explains why the current risk is high in absolute terms, it fails to explain the delta—the sudden change in the model's internal state.

A key mechanism for capturing these sudden shifts is the use of dynamic, historical baselines. In Explaining an increase in predicted risk for clinical alerts, Alvin Rajkomar et al. proposed Temporal Integrated Gradients. Instead of interpolating between zero and the current state, the method interpolates between a previous stable state and the current observation. This isolates the exact markers responsible for the sudden escalation in risk, effectively ignoring long-standing chronic conditions that were already accounted for by the model's hidden state.

Localizing Non-Stationarity and Critical Windows

In multivariate time series, feature relevance is highly non-stationary; a sensor reading might remain completely irrelevant for days before suddenly becoming the primary driver of an impending system failure.

To pinpoint these exact moments, attribution methods must explicitly quantify how much new observations alter the model's internal beliefs. As detailed by Sana Tonekaboni et al. in What went wrong and when? Instance-wise feature importance for time-series black-box models, this can be achieved by calculating the Kullback-Leibler (KL) divergence between the model's predictive distributions at consecutive time steps. By measuring the unexplained variance when specific feature subsets are withheld, their framework isolates the precise time window where a feature triggers a distributional shift in the model's predictions.

Disentangling History from Instantaneous Shocks

A major challenge in sequential models is that a feature's current influence is a mixture of two distinct signals: its long-term historical trend and its immediate, instantaneous shock.

To provide deeper insights, architectures are being designed to explicitly separate these components. For example, Kaiping Zheng et al. developed a framework in TRACER: A Framework for Facilitating Accurate and Interpretable Analytics for High Stakes Applications that decouples a feature's global, time-invariant baseline importance from its fluctuating, moment-to-moment time-variant impact.

Similarly, the authors of DeLELSTM: Decomposition-based Linear Explainable LSTM to Capture Instantaneous and Long-term Effects in Time Series decompose the hidden states of an LSTM into historical and "fresh" information tensors. This allows users to see whether a forecast anomaly was triggered by a gradual, long-term decline or a sudden, acute shock.

History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_dynamic_feature_importance_temporal_feat_971a9048,
  title = {dynamic feature importance (temporal feature importance analysis)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/dynamic-feature-importance-temporal-feature-importance-analysis/rdr_971a904890c846b5a9495add6c372cce},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_dynamic_feature_importance_temporal_feat_971a9048_v1,
  title = {dynamic feature importance (temporal feature importance analysis)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/dynamic-feature-importance-temporal-feature-importance-analysis/rdr_971a904890c846b5a9495add6c372cce/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake