Models that warn ICU clinicians when noninvasive breathing support is failing, so that a patient can be intubated before the delay itself causes harm, are usually tested only on the patient-hours where every input the model needs was actually measured. In intensive care those inputs are not measured at random. An arterial blood gas is drawn because someone is already worried, and a variable is charted only if the hospital's recording system captured it. A model that needs three or four such inputs is therefore evaluated on a subset of patients that the clinicians and the database have already filtered. That subset can be heavily enriched for patients who are deteriorating. We propose to treat "all required inputs were observed" as the intersection of two random processes, one driven by clinical suspicion and one by charting reliability. We borrow the mathematics that wireless-network theory uses for links that must pass both a shared-key test and an unreliable-channel test. The goal is a quantitative check a systematic reviewer can run from the summary statistics a paper reports. It estimates how much of the paper's reported discrimination, or lack of it, comes from this filtering rather than from the model.
The concrete test case is a puzzle that already sits in the literature. When Lijović et al. evaluated the ROX index, a bedside ratio of oxygen saturation to inspired oxygen divided by respiratory rate, the original index reached only AUROC 0.626 for failure of noninvasive ventilation (NIV), and adding arterial oxygen (PaO2) made it worse, below 0.60 (Predictive performance of ROX index and its variations for NIV failure). The same authors report that missing PaO2 stopped them from evaluating beyond 10–12 hours. We hypothesize that "adding PaO2 does not help" is partly a selection artifact. The variant that needs PaO2 is scored on a different, sicker, more homogeneous set of patient-windows. Discrimination is mechanically harder within such a set, much as a test looks weaker when everyone taking it is already at high risk.
Core Claim
The probability that a patient-window is evaluable by a model requiring a feature set $R$ is far smaller than the product of the per-feature measurement rates. The evaluable set is also systematically enriched for the outcome. Both effects can be predicted from a small number of reportable quantities: per-feature measurement rates, pairwise co-measurement rates, and how measurement rates differ between patients who later fail NIV and those who do not. We summarize the selection with an enrichment ratio
$$\rho_R = \frac{P(E_R \mid Y=1)}{P(E_R \mid Y=0)},$$
where $E_R$ is the event that every feature in $R$ was observed and charted in a window and $Y=1$ denotes NIV failure (intubation or death). We claim that $\rho_R$, together with the evaluable fraction, predicts how far a complete-case AUROC departs from the AUROC the same model would achieve on the full cohort. We further claim that an independence approximation, which multiplies marginal measurement rates, underestimates $\rho_R$ badly once $|R|$ exceeds two. The second claim is the clinical analogue of the "dangerously optimistic" connectivity estimates that come from ignoring dependence in layered random graphs.
Why Random Graph Intersection Is the Right Lens, and Where It Stops
In the key-predistribution literature, each sensor draws a random ring of keys from a shared pool. Two sensors can talk securely only if they share at least $q$ keys and their radio link is up. The resulting graph is the intersection of a random key graph with an on/off channel graph (Topological Properties of Secure Wireless Sensor Networks Under the q-Composite Key Predistribution Scheme With Unreliable Links; Performance of the Eschenauer-Gligor key distribution scheme under an ON/OFF channel). Two results from that work supply our method. First, shared draws from a common pool create dependence between edges, so independent-edge approximations mislead (Zero-One Laws for Connectivity in Random Key Graphs). Second, in heterogeneous systems the smallest key ring dominates whether the network holds together ($k$ -Connectivity of Inhomogeneous Random Key Graphs With Unreliable Links; Connectivity of Wireless Sensor Networks Secured by Heterogeneous Key Predistribution Under an On/Off Channel Model).
The mapping is direct. Each patient-window draws a "ring" of measured features from the pool of possible measurements. The ring's size depends on a latent severity level, because a worried clinician orders a bigger panel. A prediction model is a node with a fixed ring $R$. An edge between window and model, meaning the window is evaluable, requires the clinical layer to contain $R$ (a $q$-composite condition with $q=|R|$, or smaller if the model tolerates imputation). The charting layer behaves like an on/off channel whose reliability differs by database. This channel term is how we represent the granularity gap between MIMIC-IV and AmsterdamUMCdb that Lijović et al. describe.
The "smallest ring dominates" result becomes a prediction about which feature matters. The rarest required feature, almost always the blood gas, should control both the evaluable fraction and $\rho_R$. Whether a patient's trajectory stays continuously evaluable across consecutive windows is a connectivity question along the time axis. Patient-level selection bias appears when that path breaks, which is exactly what happened to Lijović et al. after 10–12 hours.
We are honest about the limits of the analogy. The zero-one laws are asymptotic in network size, and we use them only for the Poisson-style approximation of joint-observation counts. We do not claim a sharp phase transition in clinical validity. Applying intersection models to social and cybersecurity layers (Analyzing Resilience of Interest-Based Social Networks Against Node and Link Failures; SPEAR: Security Posture Evaluation using AI Planner-Reasoning on Attack-Connectivity Hypergraphs) shows the machinery travels beyond wireless networks. None of that work touches clinical measurement or bias.
Relation to Prior Work
Three groups of papers motivate this proposal. The ROX study supplies the anomaly and documents the missingness (Predictive performance of ROX index and its variations for NIV failure), but it never formalizes that missingness as a bias. Sternlicht et al. estimate a per-day mortality hazard of delayed intubation (adjusted hazard ratio 1.018) and openly flag confounding by indication (Timing to intubation COVID-19 patients: can we put it off until tomorrow?). Polok et al. show that late NIV failure, after three days, carries a worse prognosis (Noninvasive ventilation in COVID-19 patients aged ≥ 70 years—a prospective multicentre cohort study). Together these establish that the timing of the warning is what matters clinically. They also suggest that windows observed late in the waiting period are the ones most distorted by suspicion-driven measurement.
Two papers constrain the design. PROBAST+AI asks whether missing data and participant selection were handled appropriately, and it rates answers as Low, High, or Unclear risk (PROBAST+AI: an updated quality, risk of bias, and applicability assessment tool for prediction models using regression or artificial intelligence methods). The audit by Chen et al. found complete-case deletion widespread among the 51 pneumonitis models it reviewed, with only 2 rated low risk overall (Common methodological pitfalls in ICI pneumonitis risk prediction studies). Both stop at qualitative judgment. Our output is meant to plug into that existing signaling question as a number, not to replace the tool.
Finally, learned models such as cross-attention networks that encode time since last measurement (Improving Prediction of Need for Mechanical Ventilation using Cross-Attention) and the MIMIC-Extract pipeline (MIMIC-Extract: A Data Extraction, Preprocessing, and Representation Pipeline for MIMIC-III) use measurement timing as an input feature. These models exploit informative observation; they do not measure the bias it creates in evaluation, which is our question.
Research Program
The first component is a fitted generative model of joint observation. Using adult NIV episodes in MIMIC-IV and AmsterdamUMCdb, defined as in Lijović et al., we fit two layers at the level of patient-windows. The clinical layer is a latent-severity model for which features are ordered. The charting layer is a database-specific on/off reliability term, estimated from features that are recorded automatically and therefore only charting-limited. From these fits we derive closed-form or Poisson-approximated expressions for the evaluable fraction and for $\rho_R$, for any required set $R$ and imputation tolerance $q$.
The second component is a reduced version of the same calculation that uses only quantities a paper could report. These are marginal measurement rates by outcome group and pairwise co-measurement rates. This reduced version is the audit a reviewer could actually run, and it returns a predicted shift in AUROC under complete-case evaluation.
The third component is validation against ground truth. AmsterdamUMCdb's denser recording provides a subset of windows where ROX and mROX inputs are all observed. On that subset we impose observation masks sampled from the fitted MIMIC-like process and measure the resulting change in AUROC directly. This gives a semi-synthetic benchmark in which the true unselected performance is known.
Baselines
The naive reference point is complete-case evaluation as published: ROX at 10–12 hours with AUROC 0.626 and mROX variants below 0.60, as reported by Lijović et al.. Our re-extraction must first reproduce these numbers within sampling error, using identical cohort rules on both databases. If it does not, downstream comparisons are uninterpretable.
The ablation that isolates our key ingredient is the independence approximation. It is the clinical counterpart of collapsing the key graph into an Erdős–Rényi graph: it predicts evaluability and enrichment from per-feature rates multiplied together and ignores co-measurement. It gets the same inputs as our reduced audit, minus the pairwise terms. We count our method as better if its absolute error in predicted AUROC shift, measured on the masking benchmark, is at most half the independence model's error once $|R|\ge 3$.
The strongest existing correction is individual-level inverse-probability weighting for selection, with weights from a gradient-boosted model of evaluability fit on all available covariates. It is paired with multiple imputation of the missing features under the same tuning budget and the same splits. This baseline uses strictly more information than our reduced audit, so we treat it as the reference for the "true" correction on real data rather than something we expect to beat. Our goal is to match its estimated shift to within 0.02 AUROC while using only summary statistics.
There is no reported result found on Lacuna for a selection-corrected AUROC of ROX or mROX. The strongest learned model in the file, FFNN-MHA, reports AUC 0.9013 with PPV 12.04% on a different MIMIC-IV ventilation-need task (Improving Prediction of Need for Mechanical Ventilation using Cross-Attention). We include it only as a second audited model, not as a comparison target.
Why not just…?
Why not just use inverse-probability weighting or multiple imputation and skip the graph model? On individual-level data, inverse-probability weighting or multiple imputation may well correct the estimate as well as anything we build, and we expect it to. Its weakness is that a systematic reviewer applying PROBAST+AI to a published study never has the individual-level data. The contribution therefore stands or falls on whether the reduced audit, computed from reportable rates alone, tracks the weighted estimate. If it does not track within the stated tolerance, the intersection model offers nothing beyond weighting, and we will report that.
Why not just regress the AUROC shift on the overall missingness rate? A single missingness rate cannot tell apart two features that are each missing 50% of the time independently from two that are missing together. The key-graph results say this co-occurrence structure is what drives dependence. We test this directly: a one-variable regression is fit on the masking benchmark and compared with the reduced audit on held-out masking regimes and on the other database.
Why not just rely on the existing qualitative PROBAST+AI judgment? Reviewers already mark complete-case analysis as high risk, but they cannot say whether a reported null, such as "PaO2 adds nothing," is real. We will have two raters score the ROX studies and the cross-attention study qualitatively. We then check whether the quantitative audit changes any conclusion the qualitative rating would leave unclear.
Proposed Experiments and Decisive Checks
The decisive check addresses the ROX anomaly directly. We evaluate plain ROX on the mROX-evaluable windows. If ROX's AUROC falls from the full-cohort value toward the sub-0.60 mROX level, and the audit predicted that fall from $\rho_R$, then the published "PaO2 does not help" finding is largely a selection artifact. If ROX holds at its full-cohort level on that subset, the null result reflects physiology and our hypothesis about this case fails.
The second check runs the masking benchmark on AmsterdamUMCdb across required-set sizes from one to five features. We report the error in predicted AUROC shift for the full model, the reduced audit, the independence approximation, and the one-variable regression. We also verify the "rarest feature dominates" prediction by swapping in rings that differ only in the sparsest feature.
The third check tests transport. Parameters are fitted on one database and the audit is applied to the other, which tests whether the charting layer captures the granularity difference. As a final step, we apply the audit to the windows observed late in the NIV course, after more than 24 hours on support. These are the windows where Polok et al.'s delay penalty suggests early warning matters most. We then ask whether complete-case evaluation there overstates or understates how early a model could have flagged failure.
References
- Predictive performance of ROX index and its variations for NIV failure
- Topological Properties of Secure Wireless Sensor Networks Under the q-Composite Key Predistribution Scheme With Unreliable Links
- Performance of the Eschenauer-Gligor Key Distribution Scheme Under an ON/OFF Channel
- Zero-One Laws for Connectivity in Random Key Graphs
- $k$ -Connectivity of Inhomogeneous Random Key Graphs With Unreliable Links
- Connectivity of Wireless Sensor Networks Secured by Heterogeneous Key Predistribution Under an On/Off Channel Model
- Analyzing Resilience of Interest-Based Social Networks Against Node and Link Failures
- SPEAR: Security Posture Evaluation using AI Planner-Reasoning on Attack-Connectivity Hypergraphs
- Timing to intubation COVID-19 patients: can we put it off until tomorrow?
- Noninvasive ventilation in COVID-19 patients aged ≥ 70 years—a prospective multicentre cohort study
- PROBAST+AI: an updated quality, risk of bias, and applicability assessment tool for prediction models using regression or artificial intelligence methods
- Common methodological pitfalls in ICI pneumonitis risk prediction studies
- Improving Prediction of Need for Mechanical Ventilation using Cross-Attention
- MIMIC-Extract: a data extraction, preprocessing, and representation pipeline for MIMIC-III