Interpretable clinical models are usually judged on cleaned retrospective tables, and on that ground they look as good as black boxes. Bedside data are different. Vital signs come with motion artifacts, detached probes, dropouts, and hospital-specific recording habits. This proposal asks whether the reported parity between interpretable and black-box models survives that kind of data. It also proposes a specific fix. The fix is a concept bottleneck (a model that predicts through a small set of named, human-readable intermediate quantities) in which each concept is a filtered physiological state that comes with its own reliability estimate. That estimate comes from a filter that inflates its noise model when measurements stop making sense. When a sensor misbehaves, the model can then say in plain terms that it discounted the heart-rate trend because the signal was unreliable. Without the reliability estimate, the corrupted value would silently flow into a "hemodynamic instability" concept.
The second half of the contribution is about evaluation. Tracking and filtering research has a well-known habit of validating only in simulation, and clinical interpretability research is drifting the same way. We therefore treat simulated artifact injection as a tool whose own validity must be measured. For each model, we check whether its robustness ranking under injected corruption predicts its ranking under real shifts between hospitals. If it does not, simulated stress tests of interpretable models are misleading. That would be a useful negative result in its own right.
Core Claim
The accuracy-interpretability trade-off in clinical time series is not absent, as Interpretable Additive Recurrent Neural Networks For Multivariate Clinical Time Series argues from clean eICU data. Our hypothesis is that it is hidden. It should reappear as concept corruption once inputs are noisy or shifted.
Two specific failures are predicted when the concept layer has no way to represent measurement reliability. First, the concepts themselves change sharply when the input is corrupted. Second, accuracy degrades faster than for a black box trained on the same data.
Attaching filter-derived uncertainty to each concept should restore both concept stability and accuracy. It should do this without giving up the property that the final prediction is a readable function of the concepts. The new ingredient is reliability as a first-class, interpretable part of each concept. It is not the use of filtered features, which is ordinary preprocessing.
Relation to Prior Work
Two clinical bottleneck models motivate the gap.
Learning Optimal Summaries of Clinical Time-series with Concept Bottleneck Models builds sparse latent concepts from 12 hand-designed window statistics. It matches black-box LSTMs, at roughly 0.73 AUC, for predicting vasopressor onset on MIMIC-III.
I-RNN reaches the accuracy of uninterpretable recurrent models by restricting each hidden unit to a single variable. Its decay mechanism pulls unobserved variables toward population means.
Both models treat every recorded value as equally trustworthy, and both are evaluated only on curated tables. We keep their sparse, readable prediction layer. We change what flows into it, and we evaluate under corruption.
Integrating Clinical Knowledge into Concept Bottleneck Models constrains the design. It shows that aligning concepts with expert importance rankings improves out-of-domain F1 from 43.64% to 49.75% on RaabinWBC, at a small in-domain cost. This is the best existing evidence that concept structure buys robustness. It covers static images only, and we test whether the same holds for time series.
Interpretability for Time Series Transformers using A Concept Bottleneck Framework supplies two pieces of method. The first is the design choice of removing residual paths so that all information passes through the bottleneck. The second is activation patching as a causal test. Its synthetic test signals are clean sums of sines, which is exactly the kind of simulation we want to stress.
The filtering side supplies the mechanism. A Double Extended Kalman Filter Algorithm for Weakening Non-Line-of-Sight Errors in Complex Indoor Environments Based on Ultra-Wideband Technology rescales process and measurement noise covariances from measurement residuals. It does so without a hard classifier deciding whether each signal path is clear or blocked, and it stays stable where standard extended Kalman filters drift.
We borrow that residual-driven covariance scaling for physiological signals. Its validation was a single 46 m² office. Purely simulated filters such as Iterated Orthogonal Simplex Cubature Kalman Filter and Its Applications in Target Tracking show how far simulated gains of a few percent can be from deployment evidence. For that reason we do not assume that a robust filter yields robust clinical concepts; we measure it.
Research Program
For each vital sign and frequently measured lab, a per-variable state-space filter tracks two quantities: the underlying level and its rate of change. The process model is a local linear trend. The measurement noise variance is inflated whenever the normalized innovation is implausibly large, following the residual-based scaling idea of the Double Extended Kalman Filter. The innovation is the gap between what the filter expected and what was measured, scaled by the expected spread. Gaps between measurements increase the state variance in the usual way.
Each base concept therefore has three parts:
- the filtered level,
- the filtered trend,
- a variance that reflects both elapsed time and how erratic the recent measurements have been.
Higher-level concepts group a sparse set of base concepts, in the style of the summaries paper. Examples are "hemodynamic instability" and "respiratory compromise."
Base concepts enter the readable prediction layer through a precision-weighted shrinkage:
$$\tilde c_k = s_k c_k + (1 - s_k)\mu_k, \qquad s_k = \frac{\sigma_{0,k}^2}{\sigma_{0,k}^2 + P_k}$$
The symbols are defined as follows:
- $c_k$ is the filtered concept value.
- $P_k$ is the filter's variance for that concept.
- $\mu_k$ and $\sigma_{0,k}^2$ are the population mean and variance of the concept.
- $s_k \in (0,1)$ is its reliability.
The prediction is a sparse linear or logistic function of the grouped $\tilde c_k$. The explanation for any patient-hour can then be read off directly. Each concept's contribution is shown alongside its reliability. A clinician can override either the value or the reliability at test time.
The parameters that control covariance inflation and grouping are trained end to end, through the differentiable filter, against the clinical label.
Baselines
The naive reference is a logistic regression on last-observed values with forward filling. It shows how much any temporal modeling buys. On eICU hemodynamic intervention prediction, I-RNN reports 0.797 AUC for logistic regression.
The strongest interpretable competitor is I-RNN itself, which reports an eICU AUC of 0.862, matching GRU-Simple. On MIMIC-III vasopressor onset, the strongest interpretable competitor is the summary-based concept bottleneck at approximately 0.73 AUC.
The strongest unconstrained reference is a GRU with missingness indicators, which I-RNN uses as its black-box comparison. On the PhysioNet 2019 sepsis data, the reference point is the gradient-boosted ensembles that dominated the challenge. Early Prediction of Sepsis from Clinical Data: the PhysioNet/Computing in Cardiology Challenge 2019 reports that winning teams reached utility scores between 0.33 and 0.36, with drops on the held-out hospital. For PhysioNet 2012 mortality, we found no reported result on Lacuna, so we will establish our own baselines there.
Fairness rules apply to every model. All models are retrained by us on identical splits and identical preprocessing. Each gets the same hyperparameter search budget, the same artifact augmentation if any model receives it, and the same compute envelope.
The key ablation removes the uncertainty channel. It keeps the identical filter and bottleneck but fixes $s_k = 1$. This separates the contribution of reliability-carrying concepts from that of filtered features alone.
The criteria for beating the baselines differ by condition:
- On clean data, the target is not to beat anyone. Matching I-RNN within about 0.01 AUC is enough.
- Under injected artifacts and under real hospital shift, the proposal succeeds if two things hold, with confidence intervals from patient-level bootstrap. First, it degrades significantly less than I-RNN, the summary bottleneck, and the ablation. Second, its concept attributions change significantly less than theirs.
Why not just…?
Why not just train a black box and explain it with SHAP? Post-hoc attributions describe whatever the model learned, including shortcuts, and they offer no place for a clinician to correct an input-level failure before the prediction. The image results from the knowledge-aligned bottleneck work show that unconstrained models collapse out of domain. We nonetheless include a GRU with SHAP explanations under every condition, so the claim does not rest on argument alone. If its attributions turn out to be as stable as our concepts under corruption, the interpretability half of the claim fails.
Why not just smooth the signals with a standard Kalman filter and feed them to an existing bottleneck or to XGBoost? This is the most serious threat. It is exactly the fixed-$s_k$ ablation, plus an XGBoost model on filtered features. If those match our model's robustness, the contribution reduces to preprocessing, and we will report it that way. Our prediction is that they will not match. Smoothing alone cannot tell a reliable sudden drop in blood pressure from a probe artifact, whereas residual-driven covariance inflation can tell them apart.
Why not just augment a black box with simulated artifacts? Augmentation may close much of the accuracy gap, and we expect it to be competitive on AUC. We run augmented versions of every baseline. The question this cannot settle on its own is whether the explanations stay meaningful under corruption. That is decided by the concept-stability and intervention tests below.
Proposed Experiments
Training and evaluation use four datasets:
- MIMIC-III, for vasopressor onset.
- eICU, for hemodynamic intervention, with splits by hospital.
- PhysioNet 2012, for mortality.
- PhysioNet 2019, for sepsis, with training on one hospital system and testing on another. The challenge's hidden hospital data are used if accessible; otherwise the two public systems are used in both directions.
The simulation arm injects four kinds of parametrized artifacts into the clean tables:
- spike artifacts,
- flatline dropouts from probe detachment,
- biased calibration offsets,
- bursts of irregular sampling that mimic heightened clinician concern.
The realism arm uses only real shifts between hospitals, with no injected noise.
Accuracy is measured with AUC and, for sepsis, the challenge utility score. Explanation integrity is measured in three ways:
- the rank change in concept contributions between clean and corrupted copies of the same patient-hour;
- the false concept activation rate, meaning how often a corrupted input pushes a high-level concept past its alarm threshold;
- an intervention test, in which correcting the reliability of a known-corrupted concept should recover most of the lost accuracy, verified by activation patching.
The first decisive check is whether, under corruption, the uncertainty-carrying model keeps I-RNN-level accuracy while its concept contributions change significantly less than those of the fixed-reliability ablation. If the ablation does equally well, the central mechanism is refuted.
The second decisive check concerns the simulation lens. Across all models, we compute the rank correlation between degradation under injected artifacts and degradation under real hospital transfer.
- A high correlation would justify cheap simulated stress tests for clinical interpretable models.
- A low correlation would show that synthetic corruption, like the clean Gaussian sandboxes in tracking research, gives a false picture of how these models behave in deployment.
Either result tells us whether the "trade-off is a myth" conclusion holds beyond clean retrospective data.
References
- Interpretable Additive Recurrent Neural Networks For Multivariate Clinical Time Series
- Learning Optimal Summaries of Clinical Time-series with Concept Bottleneck Models
- Integrating Clinical Knowledge into Concept Bottleneck Models
- Interpretability for Time Series Transformers using A Concept Bottleneck Framework
- A Double Extended Kalman Filter Algorithm for Weakening Non-Line-of-Sight Errors in Complex Indoor Environments Based on Ultra-Wideband Technology
- Iterated Orthogonal Simplex Cubature Kalman Filter and Its Applications in Target Tracking
- Early Prediction of Sepsis from Clinical Data: the PhysioNet/Computing in Cardiology Challenge 2019