Attention Guided Policy Optimization for 3D Medical Image Registration

Jing Hu, Zhikun Shuai, Xin Wang, Shu Hu, Shanhui Sun, Siwei Lyu, Xi Wu

Abstract

Learning-based image registration approaches typically learn to map from input images to a transformation matrix. Regarding the current deep-learning-based image rigid registration approaches learn a transformation matrix in a one-shot way. Our purpose is to present a deep reinforcement learning (DRL) based method for image registration to explicitly model the step-wise nature of the human registration process. We cast an image registration process as a Markov Decision Process (MDP) where actions are defined as global image adjustment operations. Then we train our proxy to learn the optimal action sequences to achieve a good registration. More specifically, we propose a DRL proxy incorporating an attention mechanism to address the challenge of large differences in appearance between images from different modalities. Registration experiments on 3D CT-MR image pairs of patients with nasopharyngeal carcinoma and on publicly available 3D PET-MR image pairs show that our approach significantly outperforms other methods, and achieves state-of-the-art performance in multi-m-modal medical image registration.
BibTeX
@inproceedings{hu2023attention,
  author = {Jing Hu and Zhikun Shuai and Xin Wang and Shu Hu and Shanhui Sun and Siwei Lyu and Xi Wu},
  title = {{Attention Guided Policy Optimization for 3D Medical Image Registration}},
  booktitle = {IEEE Access},
  year = {2023},
  url = {https://openreview.net/forum?id=e3QHk3WNxW},
}

Loading related content...

Figure