Machine Learning Techniques for Biomedical Image Segmentation: An Overview of Technical Aspects and Introduction to State-of-Art Applications

Hyunseok Seo, Masoud Badiei Khuzani, Varun Vasudevan, Charles W. Huang, Hongyi Ren, Ruoxiu Xiao, Xiao Jia, Lei XingarXiv 2019 · DOI · OpenReview

Abstract

In recent years, significant progress has been made in developing more accurate and efficient machine learning algorithms for segmentation of medical and natural images. In this review article, we highlight the imperative role of machine learning algorithms in enabling efficient and accurate segmentation in the field of medical imaging. We specifically focus on several key studies pertaining to the application of machine learning methods to biomedical image segmentation. We review classical machine learning algorithms such as Markov random fields, k-means clustering, random forest, etc. Although such classical learning models are often less accurate compared to the deep learning techniques, they are often more sample efficient and have a less complex structure. We also review different deep learning architectures, such as the artificial neural networks (ANNs), the convolutional neural networks (CNNs), and the recurrent neural networks (RNNs), and present the segmentation results attained by those learning models that were published in the past three years. We highlight the successes and limitations of each machine learning paradigm. In addition, we discuss several challenges related to the training of different machine learning models, and we present some heuristics to address those challenges.

Introduction

Biomedical image segmentation—the process of labeling specific regions like tumors or organs in scans such as MRI or CT—is the backbone of modern computer-aided diagnosis and precision medicine. However, it is notoriously difficult due to image noise, artifacts, and "fuzzy" organ boundaries. The central challenge is the data bottleneck. While deep learning has revolutionized computer vision, medical datasets are often tiny compared to natural image sets like ImageNet. This paper explores how different machine learning paradigms handle these constraints and provides a technical overview of the architectures currently pushing the boundaries of the field.

Methodology: Classical vs. Deep Learning

The authors categorize segmentation techniques into two main eras:

1. Classical Machine Learning

Classical methods often rely on pre-specified filters and structured probabilistic models. They are particularly valuable when training data is scarce.

  • Kernel Support Vector Machines (SVMs): These use the "kernel trick" to classify pixels. To make these models computationally tractable for large images, the authors highlight the use of Random Fourier Features.
  • Random Forests & Markov Random Fields (MRFs): MRFs are particularly noted for their ability to incorporate spatial continuity, ensuring that a pixel's label is influenced by its neighbors.
Pipeline for classical machine learning using Kernel SVMs and Random Fourier Features.

2. Deep Learning (DL)

Deep learning architectures automate feature extraction, often yielding higher accuracy with sufficient data.

  • Convolutional Neural Networks (CNNs): The gold standard for automatic feature extraction.
  • Fully Convolutional Networks (FCNs) & U-Nets: These use encoder-decoder structures and "skip connections" to recover spatial resolution lost during downsampling.
  • Recurrent Neural Networks (RNNs): Used to capture sequential or morphological information.
  • Generative Adversarial Networks (GANs): Employed to generate synthetic training data to combat data scarcity.

Key Insights: The Small-Data Advantage

A critical finding is that Kernel SVMs can outperform Deep Learning (FCNs) when training data is extremely limited (e.g., training on only 1% of a dataset). Because DL models have millions of hyperparameters, they overfit quickly on small samples, whereas classical models are more "sample-efficient."

Furthermore, while 3D convolutions (V-Net) capture the best volumetric context, they are memory-intensive. Practical "2.5D" approaches using orthogonal slices serve as a middle ground for capturing 3D information.

Comparison of segmentation results or feature spaces in 3D volumetric data.

Results and Performance

The paper references wide-ranging benchmarks including BRATS (Brain Tumors), LiTS (Liver), and LIDC-IDRI (Lung). Primary metrics include the Dice Similarity Coefficient (DSC) and Intersection over Union (IoU).

  • Liver Segmentation: A modified U-Net achieved a DSC of 98.77% for liver and 86.68% for tumors, leading competition scores.
  • Expert Comparison: In cochlea segmentation, DL reached 97.8% accuracy, surpassing the 92.0% accuracy of manual expert segmentation.
  • Robustness: Incorporating MRFs as a post-processing step significantly improves the handling of noisy boundaries in CT scans.

Limitations and Future Directions

Despite their success, deep learning models remain "black boxes," hindering clinical trust. Performance is also highly sensitive to empirical heuristics like learning rates. Future work points toward Graph Neural Networks (GNNs) for non-Euclidean data, Unsupervised Learning to reduce annotation costs, and improved Transfer Learning to bridge the gap between general vision and specialized medical tasks.

Summary updated

Abstract

In recent years, significant progress has been made in developing more accurate and efficient machine learning algorithms for segmentation of medical and natural images. In this review article, we highlight the imperative role of machine learning algorithms in enabling efficient and accurate segmentation in the field of medical imaging. We specifically focus on several key studies pertaining to the application of machine learning methods to biomedical image segmentation. We review classical machine learning algorithms such as Markov random fields, k-means clustering, random forest, etc. Although such classical learning models are often less accurate compared to the deep learning techniques, they are often more sample efficient and have a less complex structure. We also review different deep learning architectures, such as the artificial neural networks (ANNs), the convolutional neural networks (CNNs), and the recurrent neural networks (RNNs), and present the segmentation results attained by those learning models that were published in the past three years. We highlight the successes and limitations of each machine learning paradigm. In addition, we discuss several challenges related to the training of different machine learning models, and we present some heuristics to address those challenges.
BibTeX
@misc{seo2019machine,
  author = {Hyunseok Seo and Masoud Badiei Khuzani and Varun Vasudevan and Charles W. Huang and Hongyi Ren and Ruoxiu Xiao and Xiao Jia and Lei Xing},
  title = {{Machine Learning Techniques for Biomedical Image Segmentation: An Overview of Technical Aspects and Introduction to State-of-Art Applications}},
  year = {2019},
  eprint = {1911.02521},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1911.02521},
}
Figure