Mask-Guided Data Augmentation for Multiparametric MRI Generation with a Rare Hepatocellular Carcinoma

Karen Sanchez, Carlos Hinojosa, Kevin Arias, Henry Arguello, Denis Kouamé, Olivier Meyrignac, Adrian Basarab

TL;DR

Researchers have developed a two-stage generative framework to synthesize high-fidelity, multiparametric MRI scans for a rare form of liver cancer. By using "mask-guided" augmentation that enforces anatomical constraints, the method produces realistic synthetic data that outperforms standard generative models, helping to bridge the data gap for rare disease research.

What Problem Does This Paper Address?

Deep learning models are data-hungry, but in the medical field—especially for rare pathologies like Massive Macrotrabecular Hepatocellular Carcinoma (MMHCC)—labeled data is incredibly scarce.

Standard data augmentation techniques (like flipping or rotating images) often fail in a clinical context. For example, flipping an abdominal scan horizontally can create "situs inversus," a condition where organs are mirrored—anatomically possible but potentially misleading for a model trained on standard anatomy. Furthermore, multiparametric MRIs (which include different phases like T1-arterial, T1-portal, and T2) require perfect spatial alignment across all sequences. Traditional GANs often struggle to maintain this global structure and the complex interrelationships between different tissues.

Methods

The authors propose a two-stage architecture designed to generate synthetic MRI triplets (T1-FS arterial, T1-FS portal, and T2) along with their corresponding tumor masks.

  1. Stage 1: Anatomical Mask Generation:
    • Tumor Transformation: Existing tumor masks are randomly transformed (zoomed, rotated, translated). Crucially, the authors use a "liver intersection mask" to ensure the synthetic tumor stays strictly within the liver boundaries.
  2. Edge Detection: They use a Canny edge detector on real MRIs to extract the "abdominal edges."
    • Synthesis: These edges and the new tumor masks are combined into a single input mask ($N_{msp}$) that serves as a structural blueprint.
  3. Stage 2: Image Synthesis via Pix2Pix:
    • The framework uses the Pix2Pix architecture, a conditional GAN (Generative Adversarial Network).
  4. Three parallel Pix2Pix networks are trained—one for each MRI phase. Because they all receive the same structural mask as input, the resulting synthetic images are perfectly co-registered, mimicking how a real patient would appear across different MRI sequences.

Datasets

The study utilized a private dataset provided by the French Society of Radiology (SFR) for a 2021 data augmentation challenge.

  • Size: 267 multiparametric MRIs from 89 patients with MMHCC.
  • Components: Each case included a tumor segmentation mask and three MRI acquisitions (T1-FS arterial, T1-FS portal, and T2-weighted).
  • Output: The authors used this small base to generate 1,000 synthetic triplets and their corresponding masks.

Key Insights and Ideas

  • Anatomical Guardrails: The "aha moment" here is the realization that GANs shouldn't have total freedom. By forcing the tumor to intersect with a manually segmented liver mask, the authors prevent the model from generating "impossible" pathologies (like a liver tumor floating in the lungs).
  • Edges as Anchors: Using Canny edges as part of the input mask provides the GAN with a "sketch" of the patient's global anatomy. This helps the model maintain the correct relationship between organs, which vanilla GANs often distort.
  • Consistency is Key: By training parallel generators on the same structural mask, the researchers solved the "multiparametric consistency" problem. The synthetic T1 and T2 images look like they belong to the same virtual patient.

Key Results

The proposed method was evaluated both quantitatively and qualitatively:

  • FID Score: The Frechet Inception Distance (FID) measures how close the synthetic distribution is to the real one (lower is better). The proposed method achieved an FID of 86.55, significantly outperforming baseline Pix2Pix (234.39) and CycleGAN (210.32).
  • Radiologist Validation: Three radiologists qualitatively evaluated the diversity and fidelity of the images using a Likert scale.
  • Competition Success: This approach was among the winners of the SFR 2021 data augmentation challenge.

Limitations and Future Work

While the results are promising, the authors note that the initial training set was quite small (89 patients). A larger, more diverse starting dataset would likely improve the "realism" of the synthetic textures even further.

The next logical step, which the authors plan to pursue, is "downstream validation." This involves testing whether adding these 1,000 synthetic cases to a training set actually improves the performance of other AI models on tasks like automatic tumor segmentation or cancer classification. Ultimately, the goal is to prove that synthetic data can effectively substitute for real-world data in training robust clinical tools.

Summary updated

Abstract

Data augmentation is classically used to improve the overall performance of deep learning models. It is, however, challenging in the case of medical applications, and in particular for multiparametric datasets. For example, traditional geometric transformations used in several applications to generate synthetic images can modify in a non-realistic manner the patients’ anatomy. Therefore, dedicated image generation techniques are necessary in the medical field to, for example, mimic a given pathology in a realistic manner. This paper introduces a new data augmentation architecture that generates synthetic multiparametric (T1 arterial, T1 portal, and T2) magnetic resonance images (MRI) of massive macrotrabecular subtype hepatocellular carcinoma with their corresponding tumor masks through a generative deep learning approach. The proposed architecture creates liver tumor masks and abdominal edges used as input in a Pix2Pix network for synthetic data creation. The efficiency of the method is demonstrated by training it on a limited multiparametric dataset of MRI triplets from 89 patients with liver lesions to generate 1, 000 synthetic triplets and their corresponding liver tumor masks. The resulting Frechet Inception Distance score was 86.55. The proposed approach was among the winners of the 2021 data augmentation challenge organized by the French Society of Radiology.
BibTeX
@inproceedings{sanchez2023mask,
  author = {Karen Sanchez and Carlos Hinojosa and Kevin Arias and Henry Arguello and Denis Kouamé and Olivier Meyrignac and Adrian Basarab},
  title = {{Mask-Guided Data Augmentation for Multiparametric MRI Generation with a Rare Hepatocellular Carcinoma}},
  booktitle = {ISBI},
  year = {2023},
  url = {https://openreview.net/forum?id=367wAmRSh6},
}

Loading related content...

Figure