Crossbar-Net: A Novel Convolutional Neural Network for Kidney Tumor Segmentation in CT Images

Qian Yu, Yinghuan Shi, Jinquan Sun, Yang Gao, Jianbing Zhu, Yakang Dai

TL;DR

This paper introduces Crossbar-Net, a novel convolutional neural network for segmenting kidney tumors in CT scans. It utilizes "crossbar patches" that capture both local and global tumor information from vertical and horizontal perspectives, and a cascaded training approach where specialized sub-models iteratively refine segmentation by focusing on difficult-to-segment regions.

What Problem Does This Paper Address?

Accurately segmenting kidney tumors in CT images is a significant challenge due to their unpredictable locations, fuzzy textures, and varied shapes. Traditional manual segmentation is time-consuming and subjective. Existing automated methods often struggle with tumors that have similar textures to surrounding tissues or exhibit internal intensity variations. This paper aims to develop a more robust and accurate automated segmentation method for kidney tumors.

Methods

The core innovation of Crossbar-Net lies in its unique approach to feature extraction and model training:

  • Crossbar Patches: Instead of standard square patches, Crossbar-Net uses "crossbar patches." These are non-squared, orthogonal patches (one vertical, one horizontal) designed to simultaneously capture both local details and global context of the tumor from different directional perspectives. This is inspired by the observation that kidney tumors often have a degree of symmetry.
  • Cascaded Training with Sub-Models: The model employs a cascaded training strategy involving two specialized sub-models: a vertical sub-model and a horizontal sub-model. These sub-models are trained iteratively. In each round, they learn from the mis-segmented regions identified by the other sub-model, allowing them to complement each other and achieve self-improvement.
  • Sampling Strategies: Two sampling strategies are introduced:
  • Basic Sampling Strategy: This strategy focuses on sampling patches near the tumor boundary and balances tumor and non-tumor samples to ensure efficient training.
  • Covering Re-sampling Strategy: This strategy specifically targets mis-segmented regions identified in the previous round, ensuring that difficult areas receive more attention in subsequent training iterations.

Datasets

The paper uses the following datasets:

  • Kidney Tumor Dataset: A custom dataset collected from 94 patients, comprising 3,500 CT slices. Each slice contains one tumor, manually annotated by physicians as ground truth. The dataset is split into training (50), validation (8), and testing (36) subjects.
  • Breast Mass Dataset: A subset of the DDSM dataset, preprocessed into PNG format. It includes 1,923 malignant and benign cases. The dataset is split into training (600), validation (100), and testing (300) images.
  • Cardiac Dataset: Publicly available MRI sequences from 33 subjects (7,980 slices) for left ventricle (LV) and myocardium segmentation. A subset of 20 subjects was used for training and validation.

Key Insights and Ideas

  • Directional Information is Crucial: The paper highlights that capturing information from specific directions (vertical and horizontal) is key to segmenting irregularly shaped and textured tumors, going beyond what standard square patches can offer.
  • Mutual Learning for Robustness: The cascaded training where sub-models learn from each other's mistakes is a powerful way to iteratively improve segmentation accuracy, especially for challenging cases.
  • Self-Improvement Mechanism: The iterative refinement process, where mis-segmented regions are re-sampled and fed back into the training, allows the model to adapt and improve its performance over multiple rounds.
  • Generalization Potential: The successful application of Crossbar-Net to cardiac and breast mass segmentation demonstrates its versatility and potential for broader medical imaging applications.

Key Results

  • Kidney Tumor Segmentation: Crossbar-Net achieved superior performance on the kidney tumor dataset, outperforming state-of-the-art methods in Dice Similarity Coefficient (DSC), True Positive Fraction (TPF), Centroid Distance (CD), and Hausdorff Distance (HD). For instance, it achieved a DSC of 0.913 and TPF of 0.915, with a significantly lower HD (8.891 mm) and CD (2.624 mm) compared to other methods.
  • Generalization: The model also showed promising results on cardiac segmentation (e.g., high DSC for MYO segmentation) and breast mass segmentation, achieving competitive results with existing methods.
  • Efficiency: The training time for kidney tumors was approximately 1 hour, and testing time was about 1.5 seconds on a regular GPU. The cascaded training, despite its iterative nature, was efficient due to strategies that reduced redundant patch sampling in later rounds.

Limitations and Future Work

  • Limitations:
    • The paper acknowledges that while Crossbar-Net is effective, the performance on cardiac segmentation was slightly better than on kidney tumors because cardiac structures (LV cavity and myocardium) are generally more regular.
  • The effectiveness of the crossbar patch strategy might be influenced by the specific aspect ratios used (e.g., 100x20 for kidney, 20x100 for cardiac).
  • Future Work:
    • The authors suggest extending the application of Crossbar-Net to other axes beyond horizontal and vertical to capture even more directional information.
  • Further exploration of its wide range of applications in medical imaging is also proposed.

Summary updated

Abstract

Due to the unpredictable location, fuzzy texture, and diverse shape, accurate segmentation of the kidney tumor in CT images is an important yet challenging task. To this end, we, in this paper, present a cascaded trainable segmentation model termed as Crossbar-Net. Our method combines two novel schemes: 1) we originally proposed the crossbar patches, which consists of two orthogonal non-squared patches (i.e., the vertical patch and horizontal patch). The crossbar patches are able to capture both the global and local appearance information of the kidney tumors from both the vertical and horizontal directions simultaneously. 2) With the obtained crossbar patches, we iteratively train two sub-models (i.e., horizontal sub-model and vertical sub-model) in a cascaded training manner. During the training, the trained sub-models are encouraged to become more focused on the difficult parts of the tumor automatically (i.e., mis-segmented regions). Specifically, the vertical (horizontal) sub-model is required to help segment the mis-segmented regions for the horizontal (vertical) sub-model. Thus, the two sub-models could complement each other to achieve the self-improvement until convergence. In the experiment, we evaluate our method on a real CT kidney tumor dataset which is collected from 94 different patients including 3500 CT slices. Compared with state-of-the-art segmentation methods, the results demonstrate the superior performance of our method on the Dice similarity coefficient, true positive fraction, centroid distance, and Hausdorff distance. Moreover, to exploit the generalization to other segmentation tasks, we also extend our Crossbar-Net to two related segmentation tasks: 1) cardiac segmentation in MR images and 2) breast mass segmentation in X-ray images, showing the promising results for these two tasks. Our implementation is released at https://github.com/Qianyu1226/Crossbar-Net.
BibTeX
@article{yu2019crossbar,
  author = {Qian Yu and Yinghuan Shi and Jinquan Sun and Yang Gao and Jianbing Zhu and Yakang Dai},
  title = {{Crossbar-Net: A Novel Convolutional Neural Network for Kidney Tumor Segmentation in CT Images}},
  journal = {IEEE Trans. Image Process.},
  year = {2019},
  url = {https://openreview.net/forum?id=p_NIOW-wB9x},
}
Figure