Post-Radiotherapy PET Image Outcome Prediction by Deep Learning under Biological Model Guidance: A Feasibility Study of Oropharyngeal Cancer Application

Hangjie Ji, Kyle J. Lafata, Yvonne M. Mowery, David Brizel, Andrea L. Bertozzi, Fang-Fang Yin, Chunhao WangarXiv 2021 · arXiv · OpenReview

TL;DR

This paper introduces a novel deep learning framework that uses a biological model to predict post-radiotherapy PET scan outcomes for oropharyngeal cancer patients. By integrating patient-specific radiation dose information into a reaction-diffusion model, the system can generate realistic time-series PET image predictions, paving the way for more personalized and adaptive radiotherapy treatment planning.

What Problem Does This Paper Address?

Predicting how a tumor will respond to radiotherapy is crucial for optimizing treatment and improving patient outcomes. Traditional methods often rely on morphological changes observed in medical images, which can be slow to manifest and may not fully capture the underlying biological processes. Functional imaging, like PET scans, offers a more direct window into metabolic activity, but accurately predicting future PET scan outcomes based on initial scans and treatment parameters remains a significant challenge. Specifically, there's a need for methods that can go beyond simple image-to-image translation and incorporate biological principles to understand and predict treatment response dynamically. This paper tackles this by aiming to predict post-radiotherapy PET images, providing insights into treatment efficacy earlier in the process.

Methods

The core of this work is a "biologically guided deep learning" framework. It combines a novel biological model with a convolutional neural network (CNN).

  • Biological Model: The researchers propose a reaction-diffusion partial differential equation (PDE) as a mathematical model for tumor response to radiation. This model extends classic reaction-diffusion systems by incorporating a "dose response term" that explicitly uses the patient's radiation dose distribution. This allows the model to capture how radiation spatially affects tumor metabolism.
  • Deep Learning Architecture: A U-Net-like encoder-decoder CNN is employed. This network is designed with two branches:
  • One branch learns the complex "response operator" (F) from the biological model, which is difficult to define analytically.
  • The second branch implements the diffusion and proliferation terms of the biological model, using trainable parameters for diffusion (α) and proliferation (β).
  • Integration: The outputs from both branches are merged according to the PDE, and this process is iterated over time steps to generate a predicted PET image sequence. This "PDE-informed" approach ensures that the deep learning model adheres to biological principles.

Datasets

The study utilized retrospective data from 64 oropharyngeal cancer patients who received curative-intent intensity-modulated radiotherapy (IMRT).

  • Data: Paired pre-radiation and post-20Gy 18FDG-PET/CT scans were collected. Radiation dose distribution maps were also obtained.
  • Data Split: For training, validation, and testing, the data was processed into 2D axial slices. Specifically, 718 slices from 38 patients were used for training, 233 slices from 13 patients for validation, and 230 slices from 13 patients for independent testing. Only slices with high 18FDG uptake (SUVmax > 1.5) in the pre-radiation scan were selected.

Key Insights and Ideas

  • Bridging Biology and Deep Learning: The most significant insight is the successful integration of a biophysical model (reaction-diffusion PDE) with a deep learning architecture. This moves beyond purely data-driven black-box models by embedding biological constraints, leading to more interpretable and potentially more robust predictions.
  • Dose-Aware Prediction: Explicitly incorporating the spatial radiation dose distribution into the biological model is a novel approach. This allows the model to learn how varying radiation levels across the tumor and surrounding tissues influence metabolic response, offering a more personalized prediction.
  • Time-Series Prediction: The framework's ability to generate time-series predictions of PET images demonstrates its potential for tracking disease evolution and response dynamically, rather than just providing a single endpoint prediction.
  • Decomposition of Response: The ability to visualize and analyze the breakdown of predicted biological effects (diffusion, proliferation, and dose response) provides valuable insights into the underlying mechanisms driving tumor changes.

Key Results

  • Accurate Prediction: The developed method successfully generated post-20Gy 18FDG-PET image outcome predictions that showed good agreement with ground-truth results.
  • Quantitative Agreement: SUV mean values in high-uptake regions of predicted images (2.45±0.25) were statistically similar to ground-truth results (2.51±0.33).
  • High Passing Rates: Gamma Index analysis, a common quality assurance metric in radiotherapy, showed high passing rates. For example, with a 10%/10mm criterion, the median passing rate was 99.9%.
  • Biological Interpretation: The breakdown of predicted biological components (diffusion, proliferation, response) provided interpretable insights into the treatment's effect on tumor metabolism.

Limitations and Future Work

  • 2D Implementation: The current model was implemented in 2D on axial slices. While this reduced computational workload and addressed data limitations, a 3D implementation is theoretically more comprehensive but would require significantly more data and computational resources.
  • Limited Data Size for 3D: The study acknowledges that a larger patient cohort would be ideal for 3D deep learning applications, but limited resources for intra-treatment imaging and the retrospective nature of the study made this challenging.
  • Operator F Interpretation: While the CNN successfully learned the dose response operator (F), its detailed mechanism and biological explanation remain unclear. Further research is needed to analytically express and biologically validate this operator.
  • Uniform Treatment Regimen: The study cohort received a uniform treatment regimen. To fully validate the model's clinical applicability, its response to different radiotherapy strategies needs to be verified.
  • Future Directions: The authors suggest future work involving small animal experiments to generate post-radiation samples with heterogeneous treatment strategies, which could help study biological models and improve deep learning intelligibility. Investigating the learned dose response term for analytical expression and biological validation is also a key area for future research.

Summary updated

Abstract

This paper develops a method of biologically guided deep learning for post-radiation FDG-PET image outcome prediction based on pre-radiation images and radiotherapy dose information. Based on the classic reaction-diffusion mechanism, a novel biological model was proposed using a partial differential equation that incorporates spatial radiation dose distribution as a patient-specific treatment information variable. A 7-layer encoder-decoder-based convolutional neural network (CNN) was designed and trained to learn the proposed biological model. As such, the model could generate post-radiation FDG-PET image outcome predictions with possible time-series transition from pre-radiotherapy image states to post-radiotherapy states. The proposed method was developed using 64 oropharyngeal patients with paired FDG-PET studies before and after 20Gy delivery (2Gy/daily fraction) by IMRT. In a two-branch deep learning execution, the proposed CNN learns specific terms in the biological model from paired FDG-PET images and spatial dose distribution as in one branch, and the biological model generates post-20Gy FDG-PET image prediction in the other branch. The proposed method successfully generated post-20Gy FDG-PET image outcome prediction with breakdown illustrations of biological model components. Time-series FDG-PET image predictions were generated to demonstrate the feasibility of disease response rendering. The developed biologically guided deep learning method achieved post-20Gy FDG-PET image outcome predictions in good agreement with ground-truth results. With break-down biological modeling components, the outcome image predictions could be used in adaptive radiotherapy decision-making to optimize personalized plans for the best outcome in the future.
BibTeX
@misc{ji2021post,
  author = {Hangjie Ji and Kyle J. Lafata and Yvonne M. Mowery and David Brizel and Andrea L. Bertozzi and Fang-Fang Yin and Chunhao Wang},
  title = {{Post-Radiotherapy PET Image Outcome Prediction by Deep Learning under Biological Model Guidance: A Feasibility Study of Oropharyngeal Cancer Application}},
  year = {2021},
  eprint = {2105.10650},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2105.10650},
}
Figure