Factors from Innovations: Finding Latent Physiological States in Autocorrelated Patient Data

Bedside risk scores such as APACHE-II and SAPS-II summarize a patient once, at admission, and lose predictive value within days. The usual fix, a recurrent neural network over the full monitoring stream, predicts better but cannot say which underlying physiological processes are failing or which one drives the others. A natural middle path is to compress dozens of vital signs and lab values into a few interpretable latent factors, in the spirit of psychometric factor analysis, and then ask how those factors influence each other over time. This note argues that the obvious version of that pipeline is broken in a specific way, and proposes a fix.

The problem is persistence. Clinical variables strongly depend on their own past values, and many of them drift together because of shared slow trends: circadian rhythm, sedation, fluid loading, the general recovery arc. Factor analysis applied to raw correlations will then group variables that merely trend together, producing factors that reflect the ward's routine rather than physiology. Causal discovery for time series met the same problem years ago and solved it by conditioning on each series' own past before testing for dependence. Our core claim is that latent factors should be discovered from innovations, meaning the part of each variable its own history cannot predict, and not from raw levels. Factors found this way should replicate across hospitals, carry a time-lagged causal graph that can be read clinically, and match the risk accuracy of opaque sequence models closely enough to be worth using.

Core Claim and What Is New

Time-lagged causal discovery provides the central technique. Momentary conditional independence (MCI) testing in PCMCI and PCMCI+, from Discovering contemporaneous and lagged causal relations in autocorrelated nonlinear time series datasets, conditions each test on the past of both the candidate cause and the target. This prevents a single causal event from showing up as dependence across many later lags, which the field calls "smearing." High-recall causal discovery for autocorrelated time series with latent confounders describes this conditioning as variance reduction and extends it to unobserved common causes (LPCMCI). Both papers fix edges between observed variables. Neither asks what latent structure the observed variables share.

Exploratory factor analysis has a graph-based form we can reuse. The Correlation Thresholding Algorithm for Exploratory Factor Analysis thresholds the correlation matrix, treats each maximal clique as the indicators of one factor, and so obtains the factor count and a zero-pattern of loadings together, which removes the arbitrary rotation step. Neuro-Causal Factor Analysis motivates cliques causally through Reichenbach's common cause principle: dependent variables share a cause. Both methods assume independent, cross-sectional samples. Under persistence, that assumption fails in a predictable direction. Two independent random walks look correlated, so the dependence graph gains spurious edges and the clique cover gains spurious or merged factors. NCFA's own reported fragility to errors in its independence tests becomes systematic error once the data are autocorrelated.

The proposal is therefore concrete. Fit each variable $x_{i,t}$ on its own lags only, giving an innovation $e_{i,t} = x_{i,t} - \hat{g}_i(x_{i,t-1},\dots,x_{i,t-p})$. Then model the contemporaneous innovation vector as

$$e_t = \Lambda \eta_t + \epsilon_t,$$

where $\Lambda$ is a sparse loading matrix whose zero pattern comes from correlation-thresholding cliques on the innovation correlation graph, $\eta_t$ are the shocks to the latent factors, and $\epsilon_t$ is variable-specific noise. We residualize on a variable's own past only, deliberately not on the full set of causal parents that PCMCI would estimate. Lagged cross-variable effects do not create same-time correlation among innovations. A shared momentary driver does. Factor scores, accumulated into factor series $f_t$, then enter PCMCI+ together with treatment indicators. The result is a lagged causal graph among latent physiological states. Granger-style predictive tests of factor-to-outcome links supply the bridge to dynamic risk.

The closest existing work is Learning Temporally Causal Latent Processes from General Temporal Data (LEAP). LEAP learns causally related latent processes end to end with a variational autoencoder and an independent-noise prior. It is a strong method, but it is a black-box encoder with no sparse indicator structure and no split-sample confirmation. InvarGC infers latent confounders, but only to remove them from Granger graphs among observed variables, not to interpret them. Our contribution is an explicit, sparse, replicable factor structure for autocorrelated data, plus a demonstration of when it matters.

Research Program

The work has three parts. The first is a simulation study that isolates the failure mode. We generate latent factor processes with known lagged causal links, each loading on a known clique of indicators, and add shared slow trends and per-indicator autoregressive persistence ranging from mild to extreme (coefficients up to 0.99, matching the stress regimes used for PCMCI+). We measure factor-count error and loading-pattern F1 for correlation thresholding on raw levels, on first differences, and on innovations, and we measure downstream graph recovery when PCMCI+ runs on the resulting factor series. The prediction is that raw-level discovery merges trend-sharing cliques and invents trend factors as persistence rises, while innovation-based discovery stays stable.

The second part is clinical factor discovery with validation in the style of psychometrics. Using hourly-binned vitals and frequently measured labs from MIMIC-IV, processed with the pipeline of An Extensive Data Processing Pipeline for MIMIC-IV, we discover factors on one cohort split. We then freeze the loading pattern and test it with confirmatory factor analysis, using fit indices RMSEA and CFI, on a held-out split and on MIMIC-III as a second, overlapping but distinct cohort. Forward-filling irregular measurements manufactures persistence, so innovations are computed only at observed timestamps, and missingness indicators are kept as separate channels and are not imputed into the factors. The INSPIRE intraoperative cohort from Beyond Thresholds offers a densely and regularly sampled setting where irregular sampling is a minor issue.

The third part connects the factors to risk. PCMCI+ and LPCMCI run on factor series plus treatments (vasopressors, fluids, sedation) to produce a lagged factor graph, read with care: in observational ICU data, treatments respond to the patient's state. The graph is reported as predictive-temporal structure with explicit bidirected confounding marks from LPCMCI, not as treatment effects. For risk, a sparse model on the current factor values and their recent lagged parents predicts 12-hour mortality and deterioration in rolling windows. Response curves in the manner of Function-Valued Causal Influence in Nonlinear Time Series then show the shape of each factor's effect on risk, such as thresholds or saturation, instead of a single importance score. That paper showed single scores can hide very different mechanisms.

Baselines

The naive reference is a static admission score: SAPS-II computed at admission and held fixed. Monitoring ICU Mortality Risk with a Long Short-Term Memory Recurrent Neural Network and Optimal intensive care outcome prediction over time using machine learning both show it decays over time. Any dynamic method must beat it at every horizon beyond day one, or the exercise is pointless.

The strongest predictive competitor is the bidirectional LSTM of Yu et al. over a 48-hour history window, predicting 12-hour forward mortality on MIMIC-III. The file reports that it substantially outperformed SAPS-II but gives no AUROC, so no reported result was found on Lacuna for that metric. We will retrain it on our exact splits, inputs, and prediction windows, with the same hyperparameter search budget as our risk model. A GRU without explicit time encoding is a second strong neural baseline, since Does Temporal Encoding Matter? found explicit time encodings gave no benefit on MIMIC-IV. We do not expect to beat these networks on AUROC. Success means staying within roughly 0.02 AUROC while giving a sparse, replicated factor structure the networks cannot provide. A gap larger than that counts as a negative result for clinical use.

The interpretable-latent competitor is the 6-state Gaussian hidden Markov model of Beyond Thresholds, which found prognostic signal even inside clinically normal ranges. We will run it on INSPIRE and on MIMIC-IV inputs, with the number of states tuned on the same validation data. Beating it means equal or better discrimination together with factors that keep their indicator structure across cohorts. Discrete HMM states have no such indicator structure to check.

LEAP is the strongest latent-discovery competitor. It reports a mean correlation coefficient of about 0.98 on synthetic latent recovery, against 0.4–0.7 for BetaVAE, iVAE, and SlowVAE (Learning Temporally Causal Latent Processes from General Temporal Data). On our synthetic data it will be scored on the same latent-recovery metric, and on clinical data on held-out risk and on cross-cohort stability of the encoder-to-variable relationship.

The key ablation keeps everything identical but runs correlation thresholding on raw levels instead of innovations. A second ablation residualizes on full PCMCI parent sets instead of own past. We expect this to strip out genuine shared signal, and the experiment will show whether it does. The correlation-thresholding paper reports a loading-pattern F1 of 0.974 on the ABCD UPPS-P scale (The Correlation Thresholding Algorithm for Exploratory Factor Analysis). That is a cross-sectional ceiling, and we will check that our implementation reproduces it before using it.

Why not just…?

Why not just run LPCMCI directly on all observed variables, since it already marks hidden common causes with bidirected edges? With 30–60 clinical variables, LPCMCI's output on shared physiology would be a dense tangle of bidirected edges that says "something common is here" without naming it. Its data requirements also grow with dimension, an open problem the file notes for this family. This is a fair alternative, so we test it. We run LPCMCI on observed variables and count how many of its bidirected edges fall inside our discovered cliques. If most do, the factors are a compact, testable summary of what LPCMCI only marks as confounding. If they do not, our factors are not capturing the confounders, and the claim weakens.

Why not just apply ordinary factor analysis or PCA to the raw series and then run PCMCI on the factor scores? This is the raw-level ablation, and it is the main threat. If moderate persistence leaves raw-level factors nearly identical to innovation factors, the residualization step adds nothing. The simulation sweep shows where along the persistence axis the two diverge, and the clinical held-out fit shows whether real ICU data sit in the regime where the difference matters. We will report the result either way.

Why not just use LEAP or a hidden Markov model, which already learn temporal latent states? Both are strong at prediction, and LEAP's synthetic recovery is excellent. Neither yields a sparse map from each latent to a named set of indicators that can be confirmed on an independent cohort, and that map is what bedside interpretation needs. If LEAP matches our cross-cohort stability and risk accuracy, the honest conclusion is that end-to-end learning suffices, and our contribution shrinks to the diagnostic simulation result.

Decisive Checks

Three checks decide the proposal. First, in simulation, raw-level factor discovery must fail measurably (wrong factor count, loading F1 well below the innovation version) at high persistence, while innovation-based discovery holds. If both degrade equally, the core claim is false. Second, the innovation factor pattern found on one MIMIC-IV split must reach acceptable confirmatory fit on the held-out split and on MIMIC-III, with fit clearly better than the raw-level pattern. Third, the sparse factor-based risk model must beat fixed SAPS-II at every horizon after day one and come within about 0.02 AUROC of the retrained LSTM. If all three hold, we have an interpretable dynamic risk model whose latent states are replicable and arranged in a readable lagged graph. If only the first holds, we still have a useful methodological warning for anyone running factor analysis on time series.