Edge-Sensitive Left Ventricle Segmentation Using Deep Reinforcement Learning

Jingjing Xiong, Lai-Man Po, Kwok-Wai Cheung, Pengfei Xian, Yuzhi Zhao, Yasar Abbas Ur Rehman, Yujia Zhang

TL;DR

This paper introduces a novel deep reinforcement learning (DRL) approach for segmenting the left ventricle (LV) in cardiac MRI images. By mimicking the human process of tracing boundaries, their DRL agent iteratively identifies edge points, achieving competitive accuracy with deep learning methods and outperforming them when training data is scarce.

What Problem Does This Paper Address?

Accurately segmenting the left ventricle (LV) in cardiac MRI is crucial for diagnosing heart conditions like mechanical dyssynchrony. Manual segmentation is time-consuming and prone to variability. While deep learning methods have shown promise, they often require large datasets. Existing reinforcement learning approaches for segmentation have struggled with accuracy due to reliance on simple thresholding. This paper aims to bridge this gap by developing a DRL agent that can perform accurate LV segmentation, even with limited training data, by learning a more human-like iterative boundary tracing process.

Methods

The core of this work is a novel DRL agent designed to mimic how a human might segment an object: by finding an initial point and then progressively tracing the boundary. This process is framed as a Markov Decision Process (MDP). The DRL agent consists of two neural networks:

  • First-P-Net: This network identifies an initial "seed" edge point on the LV boundary and generates a probability map of potential edge locations.
  • Next-P-Net: This is a Double Deep Q-Network (DQN) that takes the current state (including image information, edge features, and past point locations) and learns to predict the coordinates of the next edge point. This iterative process continues until a closed contour is formed.

The state representation is carefully designed to include grayscale image data, edge information (from a Sobel filter), a probability map of edge points, and a map of previously identified points. The reward function is a combination of difference in Intersection over Union (IoU) between current and previous segmentation masks, edge distance to the ground truth, and a penalty for point clustering to encourage smooth contours.

Datasets

The proposed model was trained and evaluated on two widely used public datasets:

  • Automated Cardiac Diagnosis Challenge (ACDC) 2017 dataset: Contains 100 cardiac MRI sequences from healthy and diseased subjects. Ground truth segmentations for LV endocardium and epicardium are available for specific phases.
  • Sunnybrook Cardiac MR Left Ventricle Segmentation Challenge-Sunnybrook 2009 dataset: Includes 45 sequences from various cardiac conditions. Hand-drawn contours for LV endocardium and epicardium are provided for End-Diastolic (ED) and End-Systolic (ES) slices.

Additionally, experiments were conducted on a smaller subset of 15 subjects from the ACDC dataset to specifically evaluate performance with limited data.

Key Insights and Ideas

  • Human-like Iterative Segmentation: The most significant insight is framing segmentation as a sequential decision-making problem, mimicking how humans trace boundaries point-by-point. This allows the DRL agent to learn a more intuitive and potentially robust segmentation strategy.
  • Edge-Centric Approach: By focusing on identifying edge points sequentially, the model directly addresses the boundary definition, which is critical for accurate segmentation.
  • Leveraging DRL for Scarce Data: The paper demonstrates that DRL can be particularly effective when training data is limited, achieving higher F-measure accuracy compared to deep learning baselines in such scenarios. This is a crucial advantage for medical imaging where data annotation is expensive.
  • Importance of State Representation: The inclusion of multiple information layers in the state (grayscale, edge, probability maps, past points) is shown to be vital for comprehensive understanding and better decision-making by the DRL agent.

Key Results

  • The proposed DRL model achieved comparable performance to state-of-the-art deep learning methods on the ACDC 2017 and Sunnybrook 2009 datasets in terms of F-measure.
  • Crucially, when trained with a very limited number of samples, the proposed model achieved higher F-measure accuracy than deep learning methods.
  • The First-P-Net was found to be accurate in locating the initial edge point, providing a good starting state for the DRL agent.
  • The iterative process of finding edge points was efficient, taking only 100-150 steps (around 0.9 seconds) per image after convergence.

Limitations and Future Work

  • Difficulty in Recovery from Errors: The authors acknowledge that if the agent deviates significantly from the correct boundary early on, it can be difficult for it to recover.
  • Limited State Information: While the state representation is rich, it could potentially be improved by incorporating more high-level information from earlier network layers.
  • Fixed Action Space: The "n skip neighborhoods" parameter is fixed. An adaptive step size for actions could offer more flexibility.
  • Exploration Policy: More sophisticated exploration policies might further enhance learning stability and efficiency.
  • Computational Load: The step-by-step nature of the DRL agent leads to a higher computational load per image compared to traditional deep learning methods, although the overall inference time is competitive.

Summary updated

Abstract

Deep reinforcement learning (DRL) has been utilized in numerous computer vision tasks, such as object detection, autonomous driving, etc. However, relatively few DRL methods have been proposed in the area of image segmentation, particularly in left ventricle segmentation. Reinforcement learning-based methods in earlier works often rely on learning proper thresholds to perform segmentation, and the segmentation results are inaccurate due to the sensitivity of the threshold. To tackle this problem, a novel DRL agent is designed to imitate the human process to perform LV segmentation. For this purpose, we formulate the segmentation problem as a Markov decision process and innovatively optimize it through DRL. The proposed DRL agent consists of two neural networks, i.e., First-P-Net and Next-P-Net. The First-P-Net locates the initial edge point, and the Next-P-Net locates the remaining edge points successively and ultimately obtains a closed segmentation result. The experimental results show that the proposed model has outperformed the previous reinforcement learning methods and achieved comparable performances compared with deep learning baselines on two widely used LV endocardium segmentation datasets, namely Automated Cardiac Diagnosis Challenge (ACDC) 2017 dataset, and Sunnybrook 2009 dataset. Moreover, the proposed model achieves higher F-measure accuracy compared with deep learning methods when training with a very limited number of samples.
BibTeX
@inproceedings{xiong2021edge,
  author = {Jingjing Xiong and Lai-Man Po and Kwok-Wai Cheung and Pengfei Xian and Yuzhi Zhao and Yasar Abbas Ur Rehman and Yujia Zhang},
  title = {{Edge-Sensitive Left Ventricle Segmentation Using Deep Reinforcement Learning}},
  booktitle = {Sensors},
  year = {2021},
  url = {https://openreview.net/forum?id=6RdHuqCXW2},
}
Figure