To form a complete diagnostic picture of complex diseases, clinicians frequently rely on multiple distinct imaging lenses. For instance, evaluating a brain tumor might require four separate MRI sequences to delineate fluid, core tumor, and cellular infiltration. Diagnosing neurodegenerative conditions often demands multiple Positron Emission Tomography (PET) scans using different radiopharmaceuticals—one to map glucose metabolism and another to track specific protein aggregations.
This sequential approach introduces a heavy multi-modal acquisition overhead. As noted by Manabu Kinoshita et al. in A Novel PET Index, 18F-FDG–11C-Methionine Uptake Decoupling Score, Reflects Glioma Cell Infiltration, multi-scan protocols face steep logistical hurdles, including compounding healthcare costs, extended clinical assessment times, and elevated cumulative radiation exposure. Furthermore, when patients move between separate imaging sessions, spatial misregistration becomes a major challenge, complicating the precise alignment of complementary diagnostic features.
To address this, a major research direction has emerged focused on workflow consolidation: eliminating the overhead of multiple physical scans by either computationally synthesizing missing modalities from a single acquisition, extracting latent parameters from routine sequences, or multiplexing signals simultaneously.
Cross-Modal Synthesis via Generative Models
One powerful strategy to eliminate acquisition overhead is cross-modal synthesis—using machine learning to computationally generate a missing modality or contrast from an existing one. Instead of physically scanning a patient multiple times, a generative model acts as a virtual scanner.
Why does this work? Medical imaging modalities are not statistically independent; they capture different physical manifestations of the same underlying biological structures. A T1-weighted MRI, a CT scan, and a PET scan all share a latent anatomical truth. Generative frameworks exploit these deep cross-modal correlations to map known inputs to hidden targets.
For example, Karim Armanious et al. introduced MedGAN: Medical image translation using GANs, which leverages specialized adversarial and perceptual losses to map images accurately across different modalities and contrasts. In a comprehensive review of the field, Sanuwani Dayarathna in Deep learning based synthesis of MRI, CT and PET: Review and analysis highlights how transitioning from standard convolutional networks to Generative Adversarial Networks (GANs) and Diffusion models allows for the creation of high-fidelity "virtual scans," drastically reducing radiation risks and costs.
This synthesis can even cross the boundary between structural and molecular imaging. Yanxi Chen et al. developed Plasma CycleGAN: Integrating Blood‐based Biomarkers for Cross‐modality Translation from MRI to PET, demonstrating that combining structural MRI with blood-based biomarkers provides enough contextual constraint to synthesize metabolic PET images. Similarly, Rui Wang et al. proposed an anatomy-aware network in Synthesizing Multi-Tracer PET Images for Alzheimer's Disease Patients using a 3D Unified Anatomy-aware Cyclic Adversarial Network to synthesize multiple target PET tracers from a single injected tracer input.
Within a single modality, the same principle allows for sequence consolidation. Xiaofeng Liu 0001 et al. developed a bijective VAE-GAN framework in Dual-Cycle Constrained Bijective Vae-Gan For Tagged-To-Cine Magnetic Resonance Image Synthesis to synthesize high-resolution cine MRI directly from tagged MRI, eliminating the need for a second scan to track cardiac motion. Similarly, Benedikt J. Schwaiger et al. demonstrated in CT-like images based on T1 spoiled gradient-echo and ultra-short echo time MRI sequences for the assessment of vertebral fractures and degenerative bone changes of the spine that specialized MRI sequences can generate "CT-like" bone structures, sparing trauma patients from undergoing an additional ionizing CT scan.
Extracting Latent Features from Single Sequences
Rather than explicitly generating an entirely new image, another facet of this research direction focuses on extracting hidden diagnostic parameters directly from standard, routine scans.
Why does this work? Standard clinical sequences often contain subtle, high-dimensional features that human eyes cannot easily quantify, but which correlate strongly with specialized, multi-sequence data. By training deep neural networks to recognize these patterns, researchers can bypass the need for specialized sequences entirely.
Consider the work of Yunlong Jin et al. in A novel MRI-based deep learning imaging biomarker for comprehensive assessment of the lenticulostriate artery-neural complex. Instead of subjecting patients to a long, expensive battery of structural, perfusion, and diffusion sequences to evaluate small-vessel health, the authors trained a deep network to extract these complex vascular properties directly from a standard T1-weighted MRI. In a similar vein, Qi Chen et al. demonstrated in A Deep Learning-Based Model for Classification of Different Subtypes of Subcortical Vascular Cognitive Impairment With FLAIR that a single FLAIR sequence contains sufficient feature density to classify cognitive impairment subtypes, bypassing multi-sequence protocols.
This concept extends to body composition and respiratory imaging:
- Sequence Reduction: Xiaomei Zhao et al. showed in A deep learning model integrating FCNNs and CRFs for brain tumor segmentation that brain tumor segmentation can be achieved using only three MRI modalities instead of the standard four, saving clinical time and storage.
- Cross-Exam Replacement: Lennard Kroll et al. showed in CT-derived body composition analysis could possibly replace DXA and BIA to monitor NET-patients that automated deep learning pipelines can extract precise muscle and fat metrics from routine clinical CT scans, rendering separate DXA or BIA exams redundant.
- Phase Elimination: For lung disease screening, Bin Chen et al. developed a model in Deep learning parametric response mapping from inspiratory chest CT scans: a new approach for small airway disease screening that computes parametric response maps using only a single inspiratory CT scan, entirely eliminating the traditional requirement for a second, high-radiation expiratory scan.
- Procedural Mode Alignment: Weijie Ma et al. utilized this concept in Toward Clinically Assisted Colorectal Polyp Recognition via Structured Cross-modal Representation Consistency by training a model to align standard white-light colonoscopy features with high-detail narrow-band imaging (NBI) data, allowing clinicians to detect subtle cues without manually toggling visualization modes.
Spatiotemporal Multiplexing and Signal Unmixing
When multiple physical scans or tracer injections are unavoidable, researchers are turning to multiplexed acquisition—capturing multiple signals simultaneously in a single session and using mathematical or deep learning models to untangle them.
Why does this work? Different biological processes or radiotracers exhibit distinct temporal dynamics or spatial distributions. By tracking the signal continuously over time (dynamic imaging), a model can separate overlapping signals based on their unique kinetic fingerprints.
In PET imaging, tracking both regional metabolism and specific protein pathology traditionally requires separate injections on different days. However, Nicolas J. Guehl et al. demonstrated in Measurement of Cerebral Perfusion Indices from the Early Phase of [18F]MK6240 Dynamic Tau PET Imaging that the first 5–10 minutes of a tau-targeted PET scan directly reflects cerebral blood flow. By capturing this "free" perfusion data before the tracer binds to tau proteins later in the scan, they achieved a comprehensive diagnostic profile from a single session. Similarly, Minyoung Oh et al. validated a dual-phase imaging protocol in Diagnostic accuracy of dual-phase 18F-FP-CIT PET imaging for detection and differential diagnosis of Parkinsonism to capture both blood flow and dopamine transporter density in one injection, while Ates Fettahoglu et al. used early-frame PET/MRI in Early-Frame [18F]Florbetaben PET/MRI for Cerebral Blood Flow Quantification in Patients with Cognitive Impairment: Comparison to an [15O]Water Gold Standard to extract perfusion metrics without a secondary scan.
Taking this further, Bailin Pan et al. developed a deep learning framework in Deep learned triple-tracer multiplexed PET myocardial image separation capable of unmixing signals from three distinct PET tracers injected simultaneously. As reviewed by Movindu Dassanayake et al. in Artificial Intelligence-Guided PET Image Reconstruction and Multi-Tracer Imaging, the advent of total-body scanners combined with AI allows clinicians to track multiple physiological functions at once, resolving the spatial misalignment and temporal separation issues inherent to traditional sequential protocols.
Hardware and Probe Co-Design
Beyond software and machine learning, workflow consolidation can be engineered directly into the imaging hardware and chemical probes.
In vascular imaging, Yasumitsu Shimizu et al. developed a hybrid system detailed in 241. Development of Rotational and Scan DSA system for Brain and Whole body that integrates a rotating C-arm with a longitudinal ceiling mount. This hardware consolidation allows clinicians to capture 3D cranial structures and wide-area peripheral blood flow simultaneously using a single injection of contrast medium, eliminating fragmented procedures.
At the molecular scale, Ann-Christin Baranski et al. tackled the disconnect between preoperative planning and intraoperative execution in PSMA-11–Derived Dual-Labeled PSMA Inhibitors for Preoperative PET Imaging and Precise Fluorescence-Guided Surgery of Prostate Cancer. By synthesizing a single "dual-labeled" probe containing both a radioactive isotope for PET and a fluorescent dye for real-time surgical guidance, the authors ensured perfect spatial consistency across the entire clinical workflow, removing the logistical complexity of administering separate diagnostic and navigational agents.
Finally, multidimensional imaging sequences can consolidate multiple clinical assessments into a single continuous acquisition. Tao Zhang demonstrated this in Feasibility of ultra-high-dimensional flow imaging for rapid pediatric cardiopulmonary MRI, where a 5D MRI technique captures anatomy, blood flow, and contrast dynamics in a single continuous scan, dramatically reducing examination times and the need for patient anesthesia.
While traditional diagnostics accept fragmented imaging sessions as a necessity, this collective body of work demonstrates that medical imaging can be consolidated. Whether through cross-modal deep learning, spatiotemporal unmixing, or hybrid hardware design, the field is moving toward a "one-stop-shop" paradigm that minimizes patient burden while maximizing diagnostic yield.
Key Foundational Work
A Novel PET Index, 18F-FDG–11C-Methionine Uptake Decoupling Score, Reflects Glioma Cell Infiltration (2012) — contributed 1 concept elements to this direction.
241. Development of Rotational and Scan DSA system for Brain and Whole body (1990) — contributed 1 concept elements to this direction.