Auxiliary Segmentation Method of Osteosarcoma MRI Image Based on Transformer and U-Net

Feng Liu, Jun Zhu, Baolong Lv, Lei Yang, Wenyan Sun, Zhehao Dai, Fangfang Gou, Jia WuComputational 2022 · DOI

TL;DR

This paper introduces OSTransnet, a novel method for segmenting osteosarcoma in MRI images by combining Transformer and U-Net architectures. It tackles challenges like image noise and blurred tumor edges, achieving improved accuracy and efficiency for clinical diagnosis and treatment.

What Problem Does This Paper Address?

Osteosarcoma, a common and aggressive bone cancer, poses significant diagnostic and treatment challenges, especially in developing countries with limited resources. Manual identification of osteosarcoma in MRI scans is time-consuming and prone to errors due to the inherent noise and blurred edges in these images. Existing automated methods struggle with these complexities, leading to potential delays in diagnosis and treatment, which negatively impacts patient prognosis. This paper aims to bridge this gap by developing a more robust and accurate automated segmentation method for osteosarcoma MRI images.

Methods

The proposed OSTransnet method is built upon a U-Net architecture, a standard for image segmentation, but with key modifications:

  • Dataset Optimization: To combat overfitting and improve robustness against noise, a novel data alignment technique is introduced. This method artificially generates more training samples by manipulating the spatial distribution of natural noise in existing images and incorporating data augmentation through image rotation.
  • Transformer Integration (CTrans): The traditional skip connections in U-Net, which can lead to a semantic gap between encoder and decoder features, are replaced by a Channeled Transformer Module (CTrans). CTrans comprises two components:
  • Channel-wise Cross Fusion Transformer (CCT): This module fuses multiscale encoder features using multi-head cross-attention, aiming to bridge the semantic gap and improve feature representation.
  • Channel-wise Cross-Attention (CCA): This component refines decoder features by focusing on channel-wise information, further enhancing semantic consistency.
  • Edge Enhancement Module (BAB): To specifically address the issue of blurred tumor edges, a Boundary Augmentation Block (BAB) is incorporated. This module uses a mask extraction algorithm and an attention mechanism to enhance edge information.
  • Combined Loss Function: A hybrid loss function is employed, combining the Dice loss (for region-based segmentation) and a Boundary loss (for edge segmentation). This dual focus helps improve both overall segmentation accuracy and the precise delineation of tumor boundaries.

Datasets

The study utilized a dataset of over 4,000 osteosarcoma MRI images provided by the Center for Artificial Intelligence Research at Monash University. The data was split into an 80% training set and a 20% test set. Data augmentation techniques, including rotation by 90, 180, and 270 degrees, were applied to the training data.

Key Insights and Ideas

  • Noise as a Feature: Instead of solely trying to remove noise, the paper cleverly uses the natural noise present in MRI scans as a source for data augmentation, creating more realistic training scenarios.
  • Bridging the Semantic Gap: Replacing U-Net's skip connections with Transformer-based modules (CTrans) is a crucial step in addressing the inherent semantic inconsistencies between different levels of feature extraction.
  • Dual Focus on Region and Edge: The combined loss function is a key innovation, recognizing that accurate segmentation requires attention to both the overall tumor area (Dice loss) and its precise boundaries (Boundary loss).
  • Edge Enhancement is Critical: The BAB module's effectiveness highlights the importance of explicitly focusing on edge information for improving segmentation accuracy in medical images where boundaries can be indistinct.

Key Results

The OSTransnet model demonstrated superior performance compared to several established segmentation methods, including FCN, PSPNet, MSFCN, MSRN, FPN, and U-Net.

  • Quantitative Improvements:
    • The model achieved a DSC of 0.949, which is approximately 6.4% higher than U-Net.
  • IOU was 0.904, 4% higher than U-Net.
  • Other metrics like Precision, Recall, and F1-score also showed significant improvements, with F1 reaching 0.951.
  • Qualitative Improvements: Visual comparisons showed that OSTransnet produced more complete and accurate segmentations, particularly for images with blurred tumor borders. The dataset optimization and edge enhancement modules were shown to significantly improve segmentation results, especially in handling fuzzy edges. The model also exhibited high stability with 98.7% reliability after 50 epochs of training.

Limitations and Future Work

  • Limitations:
    • The paper acknowledges that while the CTrans module improves feature fusion, it may still have difficulty fully capturing local information due to the reliance on cross-attention.
  • The performance of the model can be impacted by the size of the dataset, especially for small sample sizes.
  • Future Work:
    • The authors plan to further develop the method by incorporating more information to create a multiscale segmentation method. This aims to better address segmentation errors caused by subtle grayscale differences and improve overall accuracy.

Summary updated

Abstract

One of the most prevalent malignant bone tumors is osteosarcoma. The diagnosis and treatment cycle are long and the prognosis is poor. It takes a lot of time to manually identify osteosarcoma from osteosarcoma magnetic resonance imaging (MRI). Medical image processing technology has greatly alleviated the problems faced by medical diagnoses. However, MRI images of osteosarcoma are characterized by high noise and blurred edges. The complex features increase the difficulty of lesion area identification. Therefore, this study proposes an osteosarcoma MRI image segmentation method (OSTransnet) based on Transformer and U-net. This technique primarily addresses the issues of fuzzy tumor edge segmentation and overfitting brought on by data noise. First, we optimize the dataset by changing the precise spatial distribution of noise and the data-increment image rotation process. The tumor is then segmented based on the model of U-Net and Transformer with edge improvement. It compensates for the limitations of U-semantic Net by using channel-based transformers. Finally, we also add an edge enhancement module (BAB) and a combined loss function to improve the performance of edge segmentation. The method's accuracy and stability are demonstrated by the detection and training results based on more than 4,000 MRI images of osteosarcoma, which also demonstrate how well the method works as an adjunct to clinical diagnosis and treatment.
BibTeX
@inproceedings{liu2022auxiliary,
  author = {Feng Liu and Jun Zhu and Baolong Lv and Lei Yang and Wenyan Sun and Zhehao Dai and Fangfang Gou and Jia Wu},
  title = {{Auxiliary Segmentation Method of Osteosarcoma MRI Image Based on Transformer and U-Net}},
  booktitle = {Computational},
  year = {2022},
  doi = {10.1155/2022/9990092},
}
Figure