TL;DR
Electronic Health Record (EHR) data is a messy "entanglement" of patient biology and hospital care processes. This paper proves that observational data alone has a fundamental "identification limit": you cannot perfectly separate physiology from clinical decision-making using standard cross-site invariance tests or information decomposition, setting a formal boundary for what clinical AI can truly "learn" about biology.
What Problem Does This Paper Address?
When a researcher looks at a lab value in an EHR, they aren't just seeing a biological signal. They are seeing the result of a clinician's decision to order that test at a specific time for a specific patient. This is known as selective measurement or informative missingness.
The "entanglement" problem is that learned representations (like embeddings used in deep learning) compress both the patient's physiology and the institutional patterns of care. Researchers often try to resolve this using two main strategies:
- Information Decomposition: Trying to split the "value" of a lab result from the "fact that it was ordered."
- Cross-site Invariance: Comparing models across different hospitals, assuming that while hospital practices change, human biology does not.
The authors tackle a critical question: Can these methods actually identify a "universal biological mechanism" from observational data?
Methods
The authors formalize the clinical data-generating process using a Structural Causal Model (SCM), as shown in Figure 1:
Key components of their framework include:
- Latent Physiology ($Z^*$): The true, unobserved state of the patient.
- Ordering Intent ($S^*$): The clinician's unobserved decision-making process.
- Structural Undefinedness: The realization that a lab value ($V$) doesn't just "exist" as a missing value; if it wasn't ordered, it is structurally undefined.
- Observational Equivalence: A mathematical proof technique showing that two different underlying realities (one where biology is invariant across sites, and one where it isn't) can produce the exact same observed data.
They analyze these models using Partial Information Decomposition (PID) and formal proofs of identifiability.
Datasets
This is a theoretical study and does not use a real-world patient dataset. Instead, the authors use a "worked instance"—a stylized, mathematically rigorous example involving sepsis status and 30-day mortality—to demonstrate their findings. They also provide Python scripts to reproduce their numerical proofs and sensitivity analyses.
Key Insights and Ideas
- The Sentinel Trap: Analysts often use a "sentinel" code (like -1) to represent a test that wasn't performed. The paper proves that this choice forces the encoded value to carry all the information about the ordering decision, making it impossible to separate "value" from "order" using information theory (Theorem 1).
- The Invariance Illusion: A model might perform identically across two hospitals, leading researchers to believe they've captured "shared biology." However, the authors show this can be an illusion where site-specific patient mixes and hospital practices cancel each other out (Proposition 1).
- Falsification vs. Identification: Cross-site tests are useful for falsifying a hypothesis (showing that your assumptions about biology and practice don't hold up), but they cannot identify or prove a biological mechanism on their own (Theorem 2).
- Dark Matter Metaphor: Clinical judgment is the "dark matter" of the EHR. It's never recorded, yet its "gravitational effects" shape every data point we see.
Key Results
- Non-Identifiability (Theorem 3): The authors prove that biological invariance is not identifiable from observational data alone. For any model where biology is the same across sites, there exists an observationally identical model where biology is different.
- The 9.6% Gap: In their worked example, they show how a 9.6-percentage-point difference in mortality between two sites can appear even when the biological mechanism is identical. This happens because of "lossy" data representations and differences in patient populations (case mix).
- Restoring Identification: The paper isn't entirely pessimistic. It identifies "escape routes" where biology can be identified:
- Routine Measurements: When a test is performed for every patient (removing the "ordering gate"), identification becomes possible (Proposition 4).
- Multiple Imperfect Tests: Having two different tests with independent errors can restore identification in certain populations (Proposition 5).
Limitations and Future Work
The authors acknowledge several boundaries to their work:
- Fixed Encoders: The framework treats the data encoder as fixed and doesn't analyze the training process itself.
- Discrete States: Much of the detailed characterization focuses on binary or finite state spaces; continuous versions remain an open area for research.
- Treatment Effects: The model assumes the test result doesn't immediately trigger a treatment that changes the outcome independently of physiology. If it does, the identification problem becomes even more complex.
Future Directions: The authors suggest moving beyond passive observation by incorporating "identifying variation" from randomized protocol changes, "digital twins" that use mechanistic physiological models, or using LLMs to extract and account for site-specific clinical protocols.
Summary updated