A Framework for Automatic Burn Image Segmentation and Burn Depth Diagnosis Using Deep Learning

Hao Liu, Keqiang Yue, Siyi Cheng, Wenjun Li, Zhihui FuComput. Math. Methods Medicine 2021 · DOI

TL;DR

This paper introduces an end-to-end deep learning framework for automatically segmenting burn areas and diagnosing burn depth from images. By creating a new, annotated burn dataset and employing advanced convolutional neural networks, the framework aims to improve the accuracy and efficiency of burn assessment, potentially leading to better patient outcomes.

What Problem Does This Paper Address?

Burn injuries are a significant cause of morbidity and mortality, and accurate, timely diagnosis of burn extent and depth is crucial for effective treatment. Traditional methods rely on visual assessment by clinicians, which can be subjective, time-consuming, and prone to human error, especially for less experienced practitioners. While advanced imaging techniques exist, they are often expensive and not widely accessible. The authors aim to bridge this gap by developing an automated system using deep learning that can process standard camera images to provide reliable burn area segmentation and depth classification, thereby reducing diagnostic errors and costs.

Methods

The core of the proposed solution is a deep learning framework built upon fully convolutional networks (FCNs). The framework consists of an encoder-decoder architecture:

  • Encoder: Extracts feature maps from burn images. Various backbone networks are explored, including HRNetV2 and dilated ResNet variants (ResNet-50/101 with pyramid pooling module or UPerNet).
  • Decoder: Fuses and restores feature maps to the original image resolution for segmentation.

Key methodological contributions include:

  • Data Augmentation: A suite of techniques including horizontal/vertical flips, bilateral filtering, Gaussian blur, sharpening, and a pseudolabel method are employed to expand the dataset and improve model robustness.
  • Hybrid Loss Function: A combination of Cross-Entropy (CE) loss and Focal Loss (FL), further weighted by median frequency (MF) to address class imbalance, is used to train the networks.
  • End-to-End Framework: The system is designed to process raw burn images and directly output segmentation masks for burn areas and burn depths.

Datasets

The authors created a new, large-scale burn dataset by collaborating with The People's Hospital of Jianggan District, Hangzhou.

  • Initial Data: 516 unprocessed burn wound images.
  • Augmented Data: The dataset was expanded to 1200 images through cropping and resizing.
  • Annotation: Images were annotated using LabelMe software. Burn wounds were classified into five types: superficial (S), superficial partial thickness (ST), deep partial thickness (DT), full thickness (FT), and undebrided (U). The background (B) was also labeled. For simplicity in some experiments, ST and DT were merged.
  • Splits: 960 images for training and 240 for testing.

Key Insights and Ideas

  • Deep Learning for Burn Diagnosis: The paper demonstrates the potential of deep learning to automate and standardize burn diagnosis, moving beyond subjective visual assessments.
  • Importance of Data Augmentation: The study highlights that data augmentation significantly improves the performance of the segmentation networks, especially for complex tasks like multi-depth burn classification.
  • Class Imbalance Challenges: The inherent imbalance in burn depth pixel counts poses a significant challenge, requiring specialized loss functions and weighting strategies to achieve reasonable performance.
  • HRNetV2 as a Strong Backbone: The HRNetV2 architecture, when combined with a C1 module, emerged as a top performer for burn area segmentation.
  • Limitations of Current Methods for Depth: While good for burn/no-burn segmentation, achieving high accuracy in distinguishing between multiple burn depths remains challenging due to similar visual characteristics and data imbalance.

Key Results

  • Burn vs. No Burn Segmentation: The HRNetV2-C1 network achieved a best Intersection-over-Union (IOU) of 0.8467 and a mean Dice Coefficient (DC) of 0.9076 for segmenting burn versus no-burn areas.
  • Multiple Burn Depth Segmentation: For multi-burn depth segmentation, the best performing network (HRNetV2-C1 Aug & Basic_Weight) achieved a mean IOU of 0.5144, mean PA of 0.6684, and mean DC of 0.6782.
  • Impact of Loss Functions: The Total Loss (TL) function performed better than Cross-Entropy (CE) or Focal Loss (FL) alone for burn/no-burn segmentation.
  • Backbone Performance: HRNetV2-C1 generally outperformed other tested backbone networks for burn area segmentation, especially when data augmentation was applied.

Limitations and Future Work

  • Multi-Depth Segmentation Performance: The authors acknowledge that the performance for multiple burn depth segmentation is not as high as desired, with limitations in distinguishing between similar depths (e.g., deep partial thickness vs. full thickness).
  • Data Annotation Inaccuracy: Visualizing results revealed that predicted burn areas sometimes deviate from ground truth, partly due to the difficulty and subjectivity in precise edge labeling by human experts.
  • Class Imbalance Impact: The imbalance in pixel counts for different burn depths continues to be a significant hurdle, even with weighted loss strategies.

Future Work:

  • Semisupervised Learning: The authors suggest a semisupervised learning paradigm where network predictions are fed back to burn experts for refinement, creating a feedback loop to improve annotation accuracy and model performance.
  • Improved Annotation: More accurate and detailed labeling of burn datasets is crucial for further performance gains.
  • Clinical Integration: The framework's potential for clinical application in calculating Total Body Surface Area (TBSA) for burns is highlighted.

Summary updated

BibTeX
@article{liu2021framework,
  author = {Hao Liu and Keqiang Yue and Siyi Cheng and Wenjun Li and Zhihui Fu},
  title = {{A Framework for Automatic Burn Image Segmentation and Burn Depth Diagnosis Using Deep Learning}},
  journal = {Comput. Math. Methods Medicine},
  year = {2021},
  doi = {10.1155/2021/5514224},
}
Figure