Introduction
In clinical oncology, multi-phase contrast-enhanced Computed Tomography (CT) is the gold standard for tumor diagnosis. By observing the "hemodynamic profile"—how a contrast agent moves through tissues over time—radiologists can distinguish between benign and malignant lesions. Typically, this involves four phases: non-contrast (N), arterial (A), venous (V), and delayed (D).
However, real-world clinical data is often incomplete. Patients may skip phases to reduce radiation exposure, or acquisition timing may vary between hospitals, leading to significant domain shifts. Traditional deep learning models, such as U-Nets, treat these phases as fixed input channels. When a phase is missing, these models often suffer from severe performance degradation. The TARDis framework addresses this "missing modality" problem by modeling hemodynamics as a continuous process rather than discrete, independent snapshots.
Methodology: Physics-Aware Disentanglement
The core innovation of TARDis (Time Attenuated Representation Disentanglement) is treating missing modalities as missing sample points on a continuous Time-Attenuation Curve (TAC). Instead of a rigid multi-channel input, the framework employs a shared encoder followed by a dual-path architecture designed to disentangle static anatomy from dynamic perfusion.
1. Modal-Agnostic Path (Anatomy)
This branch extracts time-invariant features, such as bone and organ structures, which remain constant across all CT phases. To ensure this representation is truly agnostic to contrast enhancement, TARDis utilizes a Vector Quantization (VQ) mechanism with a learnable embedding dictionary. This acts as an information bottleneck, filtering out phase-specific "noise" (the contrast agent) and ensuring the anatomical features are consistent regardless of the input phase.
2. Modal-Specific Path (Perfusion)
This branch models the time-dependent contrast enhancement. It first regresses a relative time scalar ($\tau$) for each input scan. A Hemodynamic Conditional Variational Autoencoder (HCVAE) then learns to sample dynamic features conditioned on both the static anatomy and the estimated scan time. This allows the model to "hallucinate" missing information; for example, it can infer what an arterial phase might look like based on a non-contrast scan and learned physiological priors.
3. Adaptive Assembly
Finally, an Adaptive Assembly module uses an attention mechanism to weight and fuse these disentangled representations. This fused feature map is then passed to a decoder for the final tasks of tumor segmentation and histological subtype classification.
Experimental Results
The authors evaluated TARDis across three diverse datasets: the private Changhai (CH) kidney dataset (2,282 patients), the public C4KC-KiTS kidney dataset, and the BraTS 2018 MRI brain tumor dataset.
TARDis demonstrated significant improvements over state-of-the-art (SOTA) methods like RFNet, mmFormer, and M2FTrans, particularly in scenarios with high data sparsity.
Key Findings:
- Superior Classification: On the CH dataset, TARDis achieved a Subtype Diagnosis AUC of 0.923, significantly higher than the next best method, M2FTrans (0.752). This suggests that accurate perfusion disentanglement is critical for identifying subtle tumor subtypes.
- Robust Segmentation: Even when only a single modality (e.g., non-contrast) was available, TARDis maintained high performance (Dice score ~0.859), whereas baseline models experienced sharp drops.
- Cross-Modality Generalization: Although designed for CT hemodynamics, the model also set new SOTA results on the BraTS MRI dataset, indicating that its disentanglement strategy is robust across different imaging physics.
Conclusion
TARDis represents a shift from treating multi-modal medical imaging as a discrete engineering problem to a continuous, physics-grounded one. By explicitly modeling the underlying physiological process (the TAC), the framework achieves unprecedented robustness to missing data. Future work may extend this paradigm to other challenges, such as low-dose CT denoising and data harmonization across different clinical protocols.
Summary updated