Subsequent Boundary Distance Regression and Pixelwise Classification Networks for Automatic Kidney Segmentation in Ultrasound Images

Shi Yin, Qinmu Peng, Hongming Li, Zhengqiang Zhang, Xinge You, Susan L. Furth, Gregory E. Tasian, Yong FanarXiv 2018 · OpenReview

Abstract

It remains challenging to automatically segment kidneys in clinical ultrasound (US) images due to the kidneys' varied shapes and image intensity distributions. In this study, we propose subsequent boundary distance regression and pixel classification networks to segment the kidneys, informed by the fact that the kidney boundaries have relatively homogenous texture patterns across images. Particularly, we first use deep neural networks pre-trained for classification of natural images to extract high-level image features from US images, then these features are used as input to learn kidney boundary distance maps using a boundary distance regression network, and finally the predicted boundary distance maps are classified as kidney pixels or non-kidney pixels using a pixel classification network in an end-to-end learning fashion. We also adopted a data-augmentation method based on kidney shape registration to generate enriched training data from a small number of US images with manually segmented kidney labels. Experimental results have demonstrated that our method could effectively improve the performance of automatic kidney segmentation, significantly better than deep learning-based pixel classification networks.

What Problem Does This Paper Address?

Segmenting kidneys in clinical ultrasound (US) images is notoriously difficult. Unlike CT or MRI, ultrasound is plagued by "speckle noise" and low contrast. Furthermore, kidneys vary wildly in shape and internal texture depending on the patient's age and health condition.

A typical ultrasound image of a kidney, showing the low contrast and complex internal structures that make automatic segmentation difficult.

While semi-automatic methods exist, they are slow and suffer from high inter-operator variability—meaning two doctors might get two different results. Existing deep learning models (like the popular U-Net) often struggle because they treat every pixel inside the kidney the same way. This "pixelwise classification" approach can fail when the kidney's interior looks similar to the surrounding tissue or when boundaries are blurry. The authors argue that the boundary of the kidney actually has a more consistent texture across different patients than the interior does, making it a better target for initial feature extraction.

Methods

The authors propose a two-stage, end-to-end deep learning pipeline:

The proposed end-to-end pipeline, featuring subsequent boundary distance regression and pixelwise classification networks.
  1. Boundary Distance Regression Network: Instead of asking the AI "is this pixel a boundary?", they ask "how far is this pixel from the boundary?" They use a regression network to generate a Boundary Distance Map. This map uses a potential function to create a smooth gradient where the value is 1 at the edge and decays as you move away. This approach elegantly handles the "class imbalance" problem (where there are very few boundary pixels compared to millions of background pixels).
  2. Pixelwise Classification Network: This second network takes the distance map as input and produces the final binary mask (kidney vs. non-kidney).
  3. Multi-Loss Function: To train this end-to-end, the authors used a dynamic loss function. In the early stages of training, the model focuses heavily on getting the boundary distances right. As training progresses, the weight shifts toward perfecting the final pixel classification.

The "Secret Sauce" - Shape-Based Augmentation: Deep learning requires massive amounts of data, but medical images are scarce. The authors used Thin-Plate Splines (TPS) to register kidney shapes from one image onto the texture of another. This allowed them to generate thousands of "realistic" synthetic training samples with varied, anatomically plausible shapes, rather than just simple rotations or flips.

Key Insights and Ideas

  • Regression > Classification for Edges: Predicting a continuous distance field is more robust than binary edge detection. It provides the network with a "spatial hint" even when it's slightly off the mark.
  • Boundaries are the Anchor: The interior of a kidney can look like many things (cysts, stones, etc.), but the interface between the renal capsule and surrounding fat is a relatively stable landmark.
  • End-to-End Efficiency: Traditionally, distance maps require a slow post-processing step called a "Minimum Spanning Tree" (MST) to close the loops. By feeding the distance map into a second neural network, the authors moved that logic onto the GPU, making the process nearly instantaneous.
  • Transfer Learning Works: By starting with a VGG16 model pre-trained on natural images (like cats and dogs), the model learned high-level features much faster than starting from scratch.

Key Results

The study utilized clinical ultrasound images from the Children’s Hospital of Philadelphia (CHOP) and compared the model against benchmarks including FCNN, SegNet, U-Net, PSPNet, and DeepLabV3+.

  • Accuracy: The proposed method achieved a Dice Index of 0.9427, significantly outperforming standard U-Net (0.9245) and DeepLabV3+ (0.9311).
  • Robustness: The model showed much lower "Mean Distance" (MD) errors, meaning the predicted boundaries were physically closer to the expert-drawn ground truth.
  • Speed: The end-to-end network took only 0.18 seconds per image, compared to 3.75 seconds for the version that required manual post-processing—a 20x speedup.
  • Data Augmentation Impact: The TPS-based shape augmentation improved the performance of all tested models, proving that realistic shape variance is a critical "nutrient" for training medical AI.

Conclusion

By shifting the focus from simple interior-vs-exterior classification to boundary geometry, and employing a clever shape-based data augmentation technique, the team achieved state-of-the-art accuracy that is 20 times faster than traditional post-processing methods. While extremely low-contrast images still pose a challenge, the success of this boundary-distance logic provides a strong foundation for future extensions into 3D ultrasound volumes and more modern neural architectures.

Summary updated

Abstract

It remains challenging to automatically segment kidneys in clinical ultrasound (US) images due to the kidneys' varied shapes and image intensity distributions, although semi-automatic methods have achieved promising performance. In this study, we propose subsequent boundary distance regression and pixel classification networks to segment the kidneys, informed by the fact that the kidney boundaries have relatively homogenous texture patterns across images. Particularly, we first use deep neural networks pre-trained for classification of natural images to extract high-level image features from US images, then these features are used as input to learn kidney boundary distance maps using a boundary distance regression network, and finally the predicted boundary distance maps are classified as kidney pixels or non-kidney pixels using a pixel classification network in an end-to-end learning fashion. We also adopted a data-augmentation method based on kidney shape registration to generate enriched training data from a small number of US images with manually segmented kidney labels. Experimental results have demonstrated that our method could effectively improve the performance of automatic kidney segmentation, significantly better than deep learning-based pixel classification networks.
BibTeX
@misc{yin2018subsequent,
  author = {Shi Yin and Qinmu Peng and Hongming Li and Zhengqiang Zhang and Xinge You and Susan L. Furth and Gregory E. Tasian and Yong Fan},
  title = {{Subsequent Boundary Distance Regression and Pixelwise Classification Networks for Automatic Kidney Segmentation in Ultrasound Images}},
  year = {2018},
  eprint = {1811.04815},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1811.04815},
}
Figure