Patients who are followed over time, whether hour by hour in intensive care or year by year in a memory clinic, have the same panel of labs, scans and scores ordered again and again. Much of that repeat measurement adds little. Some biomarkers have plateaued, some drift within their normal range, and some have saturated to the point where they no longer predict anything. There is a line of machine-learning methods that decides, one patient at a time, which measurements are worth taking. They estimate how much each unobserved feature would tell us about the outcome and stop acquiring when that estimate drops below a cutoff. All of them treat the patient as a single snapshot, and all of them rely on a global cutoff that someone has to tune.
We propose a per-visit acquisition rule for longitudinal biomarkers in which the cutoff is set by the data itself. At each visit, the system scores every candidate biomarker by its predicted information about the outcome. It also scores matched "decoy" biomarkers: trajectories borrowed from other patients, which have realistic, skewed value distributions and realistic sampling patterns but by construction carry no information about this patient's outcome. A real biomarker is measured only if its score exceeds the $(1-\alpha)$ quantile of the decoy scores for this same patient at this same visit. The tuning knob then becomes $\alpha$, a false-acquisition rate with a direct meaning. It replaces a cost penalty whose scale depends on the dataset, the estimator and the hospital. The headline claim is testable: a decoy-calibrated threshold keeps its measurement budget and its accuracy when moved across hospitals and patient subgroups, where a tuned global cutoff does not.
Core Claim and What Is New
The method we build on is DIME, from Estimating Conditional Mutual Information for Dynamic Feature Selection. DIME trains a "value network" to predict, for each unobserved feature, its conditional mutual information with the label, meaning the extra information it adds given what is already known. It stops acquiring when the best candidate's predicted value falls below a cost penalty $\lambda$. DIME was evaluated on clinical tables (Intubation, ROSMAP), but only as single time slices. INVASE likewise picks patient-specific feature subsets from static vectors, on MAGGIC and PLCO. The research file names $\lambda$, and sparsity penalties like it, as the adaptive methods' remaining weakness: the rigid cutoff has been moved into a meta-parameter.
Our proposal differs from DIME in two ways. First, the value network conditions on the patient's full irregular history, and acquisition decisions are repeated at every visit, so a marker can drop out of the measurement set once it stops being informative and come back later. Second, the stopping threshold is a per-patient, per-visit null distribution produced by decoys, not a global constant. Formally, let $\hat I_{i,t}(d)$ be the value network's predicted information from measuring biomarker $d$ for patient $i$ at visit $t$, given history $H_{i,t}$. For each $d$, draw $B$ decoy trajectories $\tilde d^{(1)},\dots,\tilde d^{(B)}$ from other patients, time-aligned and keeping their missingness pattern, and compute their scores $\hat I_{i,t}(\tilde d^{(b)})$. Measure $d$ if
$$\hat I_{i,t}(d) > q_{1-\alpha}\big(\{\hat I_{i,t}(\tilde d^{(b)})\}_{b=1}^{B}\big),$$
where $q_{1-\alpha}$ is the empirical $(1-\alpha)$ quantile. Decoys have zero true information by construction. Their score distribution therefore captures how much spurious information the estimator reports for a biomarker of this shape, in this patient's context, at this point in their course. That spurious level is exactly what a fixed $\lambda$ ignores.
This rule is rank-based and makes no assumption about the shape of the distribution. That matters because A rank sum test method for informative gene discovery found that biological variables often violate normality, and that a rank-based significance cutoff sized the selected set to each dataset's signal (34 genes on Colon and 844 on Leukemia at the same $\alpha$). That paper supplies the statistical logic we borrow. Parametric thresholds such as the Gaussian feature-map model of Lee et al. and mean/median splits such as PF-PSS constrain the design: skewed, spiky biomarkers are where they should break.
Why the Longitudinal Setting Needs This
The clinical literature in the file motivates both halves of the rule. Trajectories carry information that snapshots miss. In serial albumin changes after myocardial infarction, change during admission related to mortality through an S-shaped curve. In the hsCRP trajectory classes study, a decreasing trajectory still carried an adjusted hazard ratio of 1.408 for metabolic syndrome. So the value network must see history.
A biomarker's usefulness can also collapse inside a subgroup. In the neutrophil-to-lymphocyte ratio cohort, the ratio predicted 28-day mortality in general ICU patients but not in sepsis, which the authors attribute to a ceiling effect. A global cutoff tuned on the pooled population will keep ordering such a marker for septic patients. A patient-specific null should stop, because once the marker is saturated its score looks like a decoy's.
Finally, the PhysioNet/CinC 2019 sepsis challenge documents "hospital effects", meaning different variable distributions across sites. Under those effects a $\lambda$ tuned at one hospital should buy a different number of measurements elsewhere.
The closest temporal-thresholding work is Adaptive Temporal Masking. It tracks moving averages of feature importance and thresholds against their distribution, but its "time" is training steps of a sparse autoencoder on WikiText-103, not patient follow-up. We keep its lesson, that thresholds should come from the evolving score distribution, and move it into patient time.
Research Program
The system has four parts. The first is a history encoder for irregular multivariate series. We use the dynamic local attention design from TADA because its learned per-feature windows handle both dense vitals and sparse labs. The second is a predictor head trained with random acquisition masks. The third is a DIME-style value network regressing incremental loss reduction, which DIME shows is an unbiased proxy for conditional mutual information under cross-entropy loss. The fourth is the decoy channel. During training, decoys are injected as ordinary inputs with outcome-independent values, so the value network learns to give them low scores without being told which inputs are decoys.
We will analyze the rule's statistical behavior on synthetic longitudinal data with known relevant markers, known switch points where a marker stops mattering, and heavy-tailed noise. The goal is to check whether the false-acquisition rate tracks $\alpha$ per patient and not merely on average.
The main risk is that the value network learns to recognize decoys, for example because a borrowed trajectory is inconsistent with the patient's other variables. Decoy scores would then collapse toward zero and the threshold would become too loose. We will measure this directly by training a classifier to distinguish decoys from real channels given the rest of the record. If its AUROC is well above 0.5, we fall back to decoys generated conditionally on the patient's other variables and report the degradation.
Baselines
The naive reference points bracket the problem. "Measure everything" feeds all observed biomarkers at every visit into the same TADA encoder and predictor. It sets the accuracy ceiling at 100% measurement cost; TADA itself reports AUROC 0.862 on PhysioNet 2012 and 0.817 on MIMIC-IV. A fixed-schedule baseline measures a static top-$k$ set chosen by global mutual information, with $k$ swept. It represents the fixed-budget selection that INVASE's Syn4 experiments showed failing under added noise (true positive rate around 6% for L2X versus 99.8% for INVASE).
The strongest existing method is DIME with a global $\lambda$ running on the identical encoder, predictor and value network. The only difference from our method is therefore the stopping rule. $\lambda$ gets the same tuning budget we give $\alpha$: the same grid size on the same validation split, tuned at the source hospital only. No numbers for DIME on clinical data are reported in the file ("no reported result found on Lacuna for DIME on ROSMAP or Intubation"). On static clinical data the only instance-wise selection figure is INVASE's MAGGIC AUROC of 0.720 without selection versus 0.722 with it. That figure tells us selection should not be sold as an accuracy gain, and we do not sell it that way.
A clinically motivated heuristic measures a biomarker only if its last observed change exceeded the patient's own within-person variability, estimated from their history. This is the cheap version of our idea.
Two ablations isolate the key ingredient. The first uses a pooled decoy threshold: the same decoys, but with the quantile computed over all patients, which tests whether patient-specificity matters. The second uses Gaussian decoys matched only in mean and variance, which tests whether matching skew and missingness matters.
Beating DIME means two things. At matched average cost, the area under the curve of AUROC against measurement cost must be no worse within a hospital. Across hospitals, the change in realized measurement budget and the drop in AUROC must both be significantly smaller, measured with patient-level bootstrap intervals.
Why not just…?
Why not just tune DIME's $\lambda$ carefully? This is the real threat. Within a single hospital, a well-tuned $\lambda$ may match our cost-accuracy curve, and we expect it might. Our claim rests on transfer and subgroups. The experiment tunes both knobs on PhysioNet 2019 Hospital A and freezes them for Hospitals B and C, and separately compares septic and non-septic MIMIC patients. If a frozen $\lambda$ keeps its budget and accuracy as well as a frozen $\alpha$, the method offers only interpretability, and we will say so.
Why not measure everything and let attention sort it out? Attention reweights the inputs you already have; it does not tell you what to order next, which is the question here. The measure-everything model is our ceiling. Success means staying within a small AUROC margin of it while acquiring a fraction of the measurements. It does not mean beating it.
Why not normalize each biomarker to the patient's own baseline and use a simple change rule? On slow, sparsely sampled cohorts such as ADNI this heuristic may be hard to beat, because trends are simple and there are few visits. It is included as a baseline. If it matches our method on TADPOLE but loses in the ICU, the conclusion is that learned, context-dependent thresholds matter only when many interacting markers change quickly.
Why not pick a fixed clinical panel or top-$k$ set? The rank-sum and INVASE results already show fixed sizes misallocating across settings with different signal density. The fixed-schedule baseline measures how much of that carries over to longitudinal data.
Proposed Experiments
We simulate acquisition retrospectively. A biomarker can be "acquired" only at time points where it was actually recorded, and every method faces the same candidate set, so no method gains from values that never existed.
For the ICU we use PhysioNet 2012, MIMIC-III hourly aggregates from MIMIC-Extract (which reports mortality AUROC of 0.85 to 0.90), MIMIC-IV, and the three-hospital PhysioNet 2019 data for transfer. For the 2019 data, the reported utility of top teams, 0.33 to 0.36, serves as context. For slow longitudinal follow-up we use TADPOLE, deciding which imaging, fluid and cognitive measures to collect at each visit to forecast diagnosis (scored by multiclass AUROC) and ADAS-Cog13 (scored by mean absolute error). No reported competition numbers were found on Lacuna for TADPOLE.
Three checks decide the question. The first is calibration: pure-noise biomarkers with realistic skew are injected into real records, and the per-patient rate at which each method acquires them is compared with $\alpha$. The decoy rule should stay near $\alpha$ across patient strata, while $\lambda$-thresholding's rate should vary. The second is transfer: budget drift and AUROC drop from Hospital A to B and C. The third is saturation: within septic patients, we test whether the method stops acquiring the neutrophil-to-lymphocyte ratio more often than it does in non-septic patients, consistent with the ceiling effect reported in that cohort.
We will report two sources of confounding openly. Clinicians' ordering patterns leak information, since missingness is informative. Treatments also alter trajectories. We control for the first by giving every baseline the same missingness indicators, and we treat all results as retrospective evidence rather than a validated ordering policy.
References
- Estimating Conditional Mutual Information for Dynamic Feature Selection
- INVASE: Instance-wise Variable Selection using Neural Networks
- A rank sum test method for informative gene discovery
- Stochastic Adaptive Activation Function
- PF-PSS: a double-layer parallel embedded feature selection method for cancer gene expression data
- Association between albumin changes and prognosis in older patients with acute myocardial infarction
- Association Between High-Sensitivity C-Reactive Protein Trajectories and the Incidence of Metabolic Syndrome:A Retrospective Cohort Study
- The association between the neutrophil-to-lymphocyte ratio and mortality in critical illness: an observational cohort study
- Early Prediction of Sepsis from Clinical Data: the PhysioNet/Computing in Cardiology Challenge 2019
- TIME-AWARE FEATURE SELECTION: ADAPTIVE TEMPORAL MASKING FOR STABLE SPARSE AUTOENCODER TRAINING
- Dynamic Local Attention with Hierarchical Patching for Irregular Clinical Time Series
- MIMIC-Extract: a data extraction, preprocessing, and representation pipeline for MIMIC-III
- TADPOLE Challenge: Prediction of Longitudinal Evolution in Alzheimer's Disease