3D Randomized Connection Network With Graph-Based Label Inference

Siqi Bao, Pei Wang, Tony C. W. Mok, Albert C. S. Chung

TL;DR

Researchers have developed a novel 3D deep learning framework for brain MR image segmentation that combines 3D convolutions with Convolutional LSTMs in a "Randomized Connection Network." By treating the third dimension of a scan as a temporal sequence and using probabilistic skip connections, the model achieves state-of-the-art accuracy and efficiency, further refined by a clever graph-based label inference step.

What Problem Does This Paper Address?

Segmenting sub-cortical structures (like the Hippocampus) or pathological regions (like stroke lesions) in brain Magnetic Resonance (MR) images is vital for diagnosing diseases like Alzheimer’s. However, medical images present unique challenges:

  1. Dimensionality: They are 3D volumes. Standard 2D CNNs lose "depth" information, while 3D CNNs are computationally expensive and often only capture short-range spatial dependencies.
  2. Contrast and Boundaries: Brain scans often have poor contrast, making it difficult for standard models to find precise object boundaries.
  3. Computational Load: Dense 3D prediction is slow. Traditional refinement methods like Fully Connected Conditional Random Fields (CRFs) become computationally prohibitive when dealing with millions of voxels in a 3D volume.

Methods

The authors propose a three-pronged approach:

  • Hybrid Network Units: To capture both local and global spatial-temporal features, the network uses two types of units. 3D Convolutions handle short-term, local spatial patterns. Convolutional LSTMs (ConvLSTM) treat the third axis (the slices of the brain) as a time dimension, allowing the model to remember long-term dependencies across the entire volume.
  • Randomized Connection Network: Inspired by U-Net, the architecture features symmetric contracting and expansive paths. However, unlike a static U-Net, the skip connections between these paths are randomized during training (with a probability $\alpha$). This forces the network to become an "ensemble" of multiple sub-networks, reducing layer dependency and increasing overall capacity.
  • Graph-based Label Inference: Instead of applying a heavy CRF to every pixel, the authors introduce a two-step refinement. First, Node Selection prunes "easy" pixels with high confidence. Then, Label Inference uses a Random Walker framework to focus only on the "hard" candidate pixels (usually near boundaries), significantly speeding up post-processing.

Datasets

The model was evaluated on two major benchmarks:

  • LPBA40: Contains 40 brain volumes with 56 delineated sub-cortical structures (e.g., Caudate, Putamen, Hippocampus).
  • ISLES Challenge 2016: A dataset focused on Ischemic Stroke Lesion Segmentation, which is particularly difficult due to the irregular shapes and unpredictable locations of lesions.

Key Insights and Ideas

  • The Z-Axis as Time: By applying ConvLSTM to 3D volumes, the authors effectively solve the "short-sightedness" of 3D convolution kernels without the massive memory overhead of giant kernels.
  • Dynamic DAGs: The randomized connections turn the network into a Dynamic Directed Acyclic Graph. This acts as a powerful regularizer—similar to dropout but at the architectural level—preventing the expansive path from over-relying on specific low-level feature maps.
  • Efficiency through Pruning: The realization that most voxels in a medical scan are "easy" to classify allows the graph-based inference to ignore 99.9% of the volume, focusing computational power only where it's needed (the boundaries).

Key Results

  • LPBA40 Performance: The proposed method outperformed standard FCNs and 3D U-Nets. Specifically, the randomized connection improved the Dice Coefficient (DC) by 1.26% for 3D CNNs and 2.03% for ConvLSTMs.
  • ISLES 2016: The team ranked 3rd among those who successfully completed all 19 test cases, proving the model's robustness in clinical scenarios.
  • Speedup: The graph-based label inference took only 4 seconds per structure, compared to 60 seconds for a traditional fully connected CRF, with better boundary refinement.

Limitations and Future Work

While the randomized connection improves robustness, the paper notes that ischemic stroke lesion segmentation (ISLES) remains a significant challenge for all current methods due to the extreme variability of lesions. The authors suggest that while their graph-based inference is efficient, the initial "node selection" threshold ($\theta$) is a hyperparameter that might need tuning for different types of medical images. Future work could involve automatically learning these thresholds or integrating the graph inference directly into the end-to-end training process.

Summary updated

Abstract

In this paper, a novel 3D deep learning network is proposed for brain magnetic resonance image segmentation with randomized connection, which can decrease the dependency between layers and increase the network capacity. The convolutional long-short term memory and the 3D convolution are employed as network units to capture the long-term and short-term 3D properties, respectively. To assemble these two kinds of spatial-temporal information and refine the deep learning outcomes, we further introduce an efficient graph-based node selection and label inference method. Experiments have been carried out on two publicly available databases and results demonstrate that the proposed method can obtain competitive performances as compared with the other state-of-the-art methods.
BibTeX
@article{bao2018d,
  author = {Siqi Bao and Pei Wang and Tony C. W. Mok and Albert C. S. Chung},
  title = {{3D Randomized Connection Network With Graph-Based Label Inference}},
  journal = {IEEE Trans. Image Process.},
  year = {2018},
  doi = {10.1109/tip.2018.2829263},
}

Loading related content...

Figure