Modality Propagation: Coherent Synthesis of Subject-Specific Scans with Data-Driven Regularization

Dong Hye Ye, Darko Zikic, Ben Glocker, Antonio Criminisi, Ender Konukoglu

Introduction

Medical imaging utilizes various modalities—such as T1-weighted, T2-weighted, and Diffusion Tensor Imaging (DTI)—to capture distinct anatomical and functional details. However, clinical constraints often prevent the acquisition of a complete set of modalities for every patient. This paper introduces Modality Propagation (MP), a framework designed to synthesize missing medical image modalities for specific subjects by leveraging a database of exemplar image pairs.

Methodology

The MP framework treats modality synthesis as a search-and-transfer problem, generalizing the principles of patch-based Label Propagation (LP) from discrete anatomical labels to continuous image intensity values.

Iterative Synthesis and Regularization

The core innovation of MP is its iterative, data-driven regularization. In the initial pass, the algorithm identifies matching patches in the database based solely on the available source modality (e.g., T1). In subsequent iterations, the search is refined by considering both the source image and the current estimate of the target modality. This self-correcting mechanism ensures that synthesized structures are spatially coherent and reduces noise.

Figure 4: Visual comparison of T2 and DTI-FA synthesis showing the improvement in anatomical coherence from the first to the third iteration.

Search Space Optimization

To maintain efficiency and accuracy, MP employs two primary constraints:

  1. Spatial Restriction: Matches are sought within a local window around the corresponding anatomical coordinates in aligned images.
  2. Population Restriction: The algorithm dynamically selects the $k$ most similar subjects from the database for a given region, focusing the search on relevant data.

Experimental Evaluation

The authors evaluated MP using three datasets: the NAMIC Database (20 subjects with T1, T2, and DTI-FA), the IXI Database (100 healthy subjects for reference), and the BRATS 2012 Dataset (20 brain tumor patients).

Synthesis Fidelity

Quantitative results using correlation coefficients (CC) demonstrated that MP produces high-fidelity T2 and FA maps. Notably, T2 synthesis from T1 was more accurate than FA synthesis, likely due to the higher structural similarity between T1 and T2 compared to the directionality-dependent FA maps.

"Pseudo-Healthy" Synthesis for Abnormality Detection

A compelling application of MP is the synthesis of "pseudo-healthy" scans for patients with pathologies. By synthesizing what a specific patient's brain would look like without a tumor, clinicians can generate abnormality maps by subtracting the synthesized healthy scan from the actual scan.

Figure 5: Comparison of pseudo-healthy T2 synthesis and abnormality mapping. MP-based maps are cleaner and more accurate than traditional atlas-based methods.

Discussion and Conclusion

The MP framework demonstrates that the logic of Label Propagation can be effectively extended to continuous intensity synthesis through iterative regularization. This approach is particularly advantageous for abnormality detection, as it avoids the registration errors common when warping a generic healthy atlas onto a diseased brain.

While the method shows great promise, limitations include reduced accuracy for very small anatomical structures and higher computational costs compared to direct regression models. Future work may involve expanding the training database and exploring higher-resolution synthesis.

Summary updated

Abstract

We propose a general database-driven framework for coherent synthesis of subject-specific scans of desired modality, which adopts and generalizes the patch-based label propagation (LP) strategy. While modality synthesis has received increased attention lately, current methods are mainly tailored to specific applications. On the other hand, the LP framework has been extremely successful for certain segmentation tasks, however, so far it has not been used for estimation of entities other than categorical segmentation labels. We approach the synthesis task as a modality propagation, and demonstrate that with certain modifications the LP framework can be generalized to continuous settings providing coherent synthesis of different modalities, beyond segmentation labels. To achieve high-quality estimates we introduce a new data-driven regularization scheme, in which we integrate intermediate estimates within an iterative search-and-synthesis strategy. To efficiently leverage population data and ensure coherent synthesis, we employ a spatio-population search space restriction. In experiments, we demonstrate the quality of synthesis of different MRI signals (T2 and DTI-FA) from a T1 input, and show a novel application of modality synthesis for abnormality detection in multi-channel MRI of brain tumor patients.
BibTeX
@inproceedings{ye2013modality,
  author = {Dong Hye Ye and Darko Zikic and Ben Glocker and Antonio Criminisi and Ender Konukoglu},
  title = {{Modality Propagation: Coherent Synthesis of Subject-Specific Scans with Data-Driven Regularization}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2013},
  doi = {10.1007/978-3-642-40811-3\_76},
}
Figure