TL;DR
This paper introduces causal graph fusion, a method to learn unified causal structures from fragmented, incomplete clinical datasets, and causal attention steering, which injects these learned graphs into frozen tabular foundation models at inference time. By iterating between causal discovery and Markov-blanket-guided imputation, the authors turn data fragmentation into a source of evidence, improving both structural recovery and model robustness to distribution shifts without the need for retraining.
What Problem Does This Paper Address?
Clinical knowledge is rarely found in a single, perfect dataset. Instead, it’s scattered across various cohorts—one might track nutritional factors, another neurological ones—with overlapping but non-identical variables and significant missing data.
Standard machine learning and causal discovery tools typically require a single, complete table. When faced with fragmented data, researchers often resort to naive pooling or off-the-shelf imputation, both of which can distort the underlying causal relationships. Furthermore, even if a causal graph is known, state-of-the-art "tabular foundation models" (like TabPFN) are usually "black boxes" that cannot easily incorporate this prior knowledge without expensive retraining.
Methods
The authors propose a two-stage framework:
- Causal Graph Fusion (Figure 1): Instead of imputing before discovery, this method handles missingness during discovery.
- It first learns a "pooled-shared" Partial Ancestral Graph (PAG) on variables common to all cohorts.
- This shared graph, combined with temporal background knowledge (e.g., "admission variables cannot be caused by discharge outcomes"), constrains the learning of individual per-cohort graphs.
- Crucially, it uses an iterative loop: the current Markov blanket of a variable guides its re-imputation, which in turn refines the next iteration of graph discovery.
- Causal Attention Steering (Figure 5): This technique allows a learned causal graph to "steer" a frozen foundation model (TabPFN) at inference time.
- The learned PAG is compiled into an attention mask.
- This mask is added as a scalar bias ($\gamma$) to the model's self-attention logits.
- This nudges the model to prioritize causally relevant features (like ancestors or the Markov blanket) and ignore spurious correlations, making it more robust to distribution shifts.
Datasets
The framework was validated using:
- Synthetic Benchmarks: Random Directed Acyclic Graphs (DAGs) with 6–30 nodes, tested under various missingness mechanisms (MCAR, MAR, MNAR).
- Sachs Protein-Signaling Benchmark: A classic causal discovery dataset with 11 nodes and a known ground-truth graph.
- Real-World Clinical Data: Two cohorts from the Swiss Paraplegic Center focusing on spinal cord injury (SCI):
- HAPI ($n=250$): Hospital-acquired pressure injury.
- PPIS ($n=266$): Post-pressure-injury surgery.
Key Insights and Ideas
- Fragmentation as a Feature, Not a Bug: By aligning overlapping variables, the fusion process can resolve causal orientations that no single cohort could identify on its own. For example, it successfully oriented age, albumin, and hemoglobin as ancestors of hospital length of stay (LoS) by pooling evidence.
- The Imputation-Discovery Synergy: Traditional imputation is "causally blind." By letting the evolving causal graph (specifically the Markov blanket) guide imputation, the authors achieved significantly lower distributional distortion (Figure 2).
- Inference-Time Steering: You don't need to retrain a foundation model to give it "causal common sense." Biasing the attention mechanism is a lightweight, training-free way to inject domain knowledge.
- Shift-Matched Robustness: The authors found that the "best" causal features to attend to depend on the type of shift: steer toward causal ancestors when the underlying mechanism changes, but toward the full Markov blanket when only the feature distributions (covariates) shift (Figure 6).
Key Results
- Structural Recovery: On synthetic data, fusion dominated baselines across all missingness conditions, raising the graph F1 score from ~0.63 to 0.71 (Figure 2).
- Distributional Fidelity: Fusion reduced distributional distortion (KS distance) by a third compared to non-fusion baselines (0.034 vs 0.051).
- Predictive Performance: In the SCI cohorts, the fused Markov blankets matched or exceeded automatic feature selection methods on four out of five outcomes. For secondary pressure-injury complications, the causal blanket was the best feature set overall (AUROC 0.776).
- Robustness to Shift: Causal steering significantly improved discrimination under distribution shift ($+0.011$ to $+0.029$ AUROC), a gain that random graphs failed to produce (Figure 6).
Limitations and Future Work
- Sample Size Sensitivity: Learning high-dimensional graphs from a few hundred patients remains challenging and sensitive to sampling variability.
- Temporal Resolution: While the models are temporally organized, the lack of exact onset timestamps for some outcomes means they are currently "temporally-constrained retrospective classifiers" rather than fully prospective predictors.
- Generalizability: The clinical results are from a single specialized center. External, multi-site validation is essential to confirm the transportability of the learned SCI structures.
- Model Specificity: The current steering implementation is tuned for the TabPFN v2 architecture; adapting it to newer versions (like v2.5) requires further engineering of the "thinking-token" pathways.
Summary updated