End-to-end learning of convolutional neural net and dynamic programming for left ventricle segmentation

Nhat M. Nguyen, Nilanjan Ray

Abstract

Deep learning models, particularly Convolutional Neural Networks (CNNs), have set new benchmarks in medical image segmentation but often require large annotated datasets and lack anatomical consistency. This paper introduces EDPCNN, a hybrid framework that integrates the feature extraction capabilities of CNNs with the structural constraints of Dynamic Programming (DP). To overcome the non-differentiability of DP, the authors employ a "Synthetic Gradient" approach using a differentiable proxy network. Experimental results on the ACDC dataset demonstrate that EDPCNN significantly outperforms standard CNNs in data-scarce scenarios, achieving a Dice score of 0.907 with only 10 training images.

Introduction

Medical image segmentation is critical for cardiac health assessment, yet expert-labeled data is often expensive and scarce. While CNNs like U-Net excel at pixel-level classification, they frequently produce anatomically implausible results—such as disconnected blobs or jagged edges—when training data is limited. Conversely, classical algorithms like Dynamic Programming (DP) can enforce global shape priors (e.g., smooth, closed contours) but are typically isolated from the deep learning process due to their non-differentiable nature.

This work proposes a method to bridge this gap, allowing a CNN to "learn" through a DP module. The resulting EDPCNN architecture integrates anatomical common sense directly into the training loop, providing a robust solution for left ventricle (LV) segmentation.

Methodology

The EDPCNN pipeline consists of several integrated stages designed to transform raw MRI data into a structured anatomical contour.

Architecture Overview

The process begins with a CNN (U-Net) that processes the input MRI to generate a feature "output map." This map represents the likelihood of a pixel belonging to the target boundary.

Overview of the EDPCNN architecture

Star Pattern and Warping

To leverage the circular nature of the left ventricle, a "star pattern" of radial lines is overlaid on the output map. Values are sampled along these lines to create a "Warped Map," effectively converting the circular segmentation task into a path-finding problem in a rectangular coordinate space.

Radial sampling using a star pattern

End-to-End Training via Synthetic Gradients

The core challenge of this hybrid approach is that the DP module involves an argmin operation, which is non-differentiable. To enable backpropagation, the authors introduce an Approximate Neural Network. This proxy network is trained to mimic the output of the DP module. During the training of the main CNN, gradients are passed through this differentiable proxy, a technique referred to as "Synthetic Gradients." This allows the CNN to adjust its weights based on how well the final DP-derived contour matches the ground truth. To ensure robustness, random noise is added to the Warped Map during training, forcing the proxy network to learn a more generalized representation of the DP function.

Experimental Results

The model was evaluated using a modified version of the ACDC (Automated Cardiac Diagnosis Challenge) dataset, focusing on its performance across varying training set sizes.

Data Efficiency

The most significant advantage of EDPCNN is its performance with minimal data. When trained on only 10 images, EDPCNN achieved a Dice score of 0.907, compared to just 0.695 for a standard U-Net. This suggests that the DP module acts as a powerful regularizer, preventing the CNN from overfitting to noise and forcing it to focus on anatomically relevant features.

Robustness and Accuracy

Beyond overlap metrics, EDPCNN showed superior performance in boundary-specific metrics like Average Symmetric Surface Distance (ASSD) and Hausdorff Distance (HD). The hybrid model also exhibited lower variance in results, indicating higher reliability and consistency across different patient scans compared to standalone CNN architectures.

Discussion and Conclusion

The EDPCNN framework demonstrates that classical algorithmic knowledge and modern deep learning are most powerful when combined. By using proxy networks to handle non-differentiable steps, researchers can incorporate complex shape priors and structural constraints that were previously inaccessible to standard backpropagation.

While the current implementation relies on a predefined center point for the star pattern, the authors aim to automate this localization in future iterations. The "Synthetic Gradient" approach serves as a general blueprint for incorporating other traditional computer vision tools into modern neural architectures, potentially revolutionizing segmentation tasks across various medical domains.

Summary updated

Abstract

Differentiable programming is able to combine different functions or programs in a processing pipeline with the goal of applying end-to-end learning or optimization. A significant impediment is the non-differentiable nature of some algorithms. We propose to use synthetic gradients (SG) to overcome this difficulty. SG uses the universal function approximation property of neural networks. We apply SG to combine convolutional neural network (CNN) with dynamic programming (DP) in end-to-end learning for segmenting left ventricle from short axis view of heart MRI. Our experiments show that end-to-end combination of CNN and DP requires fewer labeled images to achieve a significantly better segmentation accuracy than using only CNN.
BibTeX
@inproceedings{nguyen2020end,
  author = {Nhat M. Nguyen and Nilanjan Ray},
  title = {{End-to-end learning of convolutional neural net and dynamic programming for left ventricle segmentation}},
  booktitle = {MIDL},
  year = {2020},
  url = {https://openreview.net/forum?id=NFjz7WUYmG},
}
Figure