Chronic Wound Image Augmentation and Assessment Using Semi-Supervised Progressive Multi-Granularity EfficientNet

Ziyang Liu, Emmanuel Agu, P.c. Pedersen, Clifford Lindsay, Bengisu Tulu, Diane M. StrongIEEE Open Journal of Engineering in Medicine and Biology 2023 · DOI

TL;DR

Researchers have developed SS-PMG-EfficientNet, a deep learning framework that achieves ~90% accuracy in grading chronic wounds by combining semi-supervised learning with a "progressive multi-granularity" approach. By leveraging a large set of unlabeled images to supplement a small labeled dataset, the model outperformed previous state-of-the-art methods and revealed a surprising insight: traditional GAN-based image synthesis actually hindered performance in this fine-grained medical task.

What Problem Does This Paper Address?

Chronic wounds (diabetic ulcers, pressure sores, etc.) affect over 6.5 million people in the US alone, costing the healthcare system over $25 billion annually. Proper treatment requires accurate, consistent grading, typically done using the Photographic Wound Assessment Tool (PWAT).

However, manual grading is a significant burden for nurses and is often subjective. While deep learning offers a solution for automated assessment, it faces two major hurdles in the medical domain:

  1. Data Scarcity and Imbalance: High-quality, expert-labeled medical images are expensive and rare.
  2. Fine-Grained Complexity: Distinguishing between different types of necrotic tissue or granulation levels requires looking at extremely subtle textures and colors that standard convolutional neural networks (CNNs) often miss.

Methods

The authors propose a novel architecture called Semi-Supervised Progressive Multi-Granularity EfficientNet (SS-PMG-EfficientNet). The "secret sauce" lies in three integrated components:

  • EfficientNet B0 Backbone: A computationally efficient CNN that serves as the foundation.
  • Semi-Supervised Learning (SS): To combat the small size of their labeled dataset, the authors used a large secondary corpus of unlabeled wound images. They employed an unsupervised loss function that acts as a regularization term, allowing the model to learn the general visual features of wounds from the unlabeled data without needing explicit scores.
  • Progressive Multi-Granularity (PMG): This is the most innovative part of the pipeline. It uses a jigsaw puzzle generator to shuffle image patches at different scales. This forces the model to learn "fine-grained" features (like specific tissue textures) at small scales before progressively moving to "coarse-grained" features (like wound shape and size) at larger scales.

Datasets

The study utilized two primary sources:

  • WoundNet (Labeled): A small but high-quality dataset of 1,639 images across four wound types (diabetic foot, pressure, vascular, and surgical ulcers). Each image was labeled by experts with 8 PWAT sub-scores.
  • DFUC 2021 (Unlabeled): A much larger secondary dataset of 9,870 images from the Diabetic Foot Ulcers Grand Challenge. These were used for the semi-supervised training component.

Key Insights and Ideas

  • The "GAN Failure" Paradox: One of the most striking findings was that generating synthetic wound images using GANs (like pix2pixHD) actually decreased classification performance. While the synthetic images looked realistic to the human eye, they likely lacked the precise, high-resolution textural details (fine-grained features) necessary for clinical grading.
  • Real Unlabeled Data > Synthetic Data: The authors demonstrated that using real, unlabeled images from a related domain (DFUC) provided better "augmentation" than generating fake images. This suggests that for specialized medical tasks, the underlying distribution of real tissue is too complex for current GANs to replicate faithfully.
  • Granularity Mimics Expertise: The PMG approach effectively mimics how a human expert works—zooming in to identify tissue types (granulation vs. necrosis) before zooming out to assess the overall wound edges and size.

Key Results

  • High Accuracy: The SS-PMG-EfficientNet achieved an average accuracy and F1 score of approximately 90% across all 8 PWAT sub-scores.
  • Significant Improvement: This represents a 7% improvement over the previous state-of-the-art (Patch Attention DenseNet) without requiring any additional labeled data.
  • Efficiency: Despite the added complexity of the PMG and SS components, the model remains fast. Training took roughly 30 minutes on a single GTX 1080 Ti, compared to 4 hours for some heavier ResNet-based variants.
  • Robustness: Confusion matrices showed that the model's predictions were tightly clustered around the diagonal, meaning even when it was wrong, it was usually only off by a single grade level.

Limitations and Future Work

While the results are impressive, the authors noted a few areas for improvement:

  • Subscore 3 (Necrotic Type): This specific attribute had the lowest sensitivity (~80%). Distinguishing between different types of dead tissue remains the most challenging part of the visual assessment.
  • Class Imbalance: Some PWAT scores are naturally rarer than others. The authors suggest exploring Focal Loss in future iterations to give more weight to these "harder" or rarer examples.
  • Clinical Integration: The ultimate goal is a smartphone-based system for home use. Future work will likely focus on how these models handle the variable lighting and angles of real-world consumer photography.

Summary updated

Abstract

<i>Goal:</i> Augment a small, imbalanced, wound dataset by using semi-supervised learning with a secondary dataset. Then utilize the augmented wound dataset for deep learning-based wound assessment. <i>Methods:</i> The clinically-validated Photographic Wound Assessment Tool (PWAT) scores eight wound attributes: Size, Depth, Necrotic Tissue Type, Necrotic Tissue Amount, Granulation Tissue type, Granulation Tissue Amount, Edges, Periulcer Skin Viability to comprehensively assess chronic wound images. A small corpus of 1639 wound images labeled with ground truth PWAT scores was used as reference. A Semi-Supervised learning and Progressive Multi-Granularity training mechanism were used to leverage a secondary corpus of 9870 unlabeled wound images. Wound scoring utilized the EfficientNet Convolutional Neural Network on the augmented wound corpus. <i>Results:</i> Our proposed Semi-Supervised PMG EfficientNet (SS-PMG-EfficientNet) approach estimated all 8 PWAT sub-scores with classification accuracies and F1 scores of about 90% on average, and outperformed a comprehensive list of baseline models and had a 7% improvement over the prior state-of-the-art (without data augmentation). We also demonstrate that synthetic wound image generation using Generative Adversarial Networks (GANs) did not improve wound assessment. <i>Conclusions:</i> Semi-supervised learning on unlabeled wound images in a secondary dataset achieved impressive performance for deep learning-based wound grading.
BibTeX
@article{liu2023chronic,
  author = {Ziyang Liu and Emmanuel Agu and P.C. Pedersen and Clifford Lindsay and Bengisu Tulu and Diane M. Strong},
  title = {{Chronic Wound Image Augmentation and Assessment Using Semi-Supervised Progressive Multi-Granularity EfficientNet}},
  journal = {IEEE Open Journal of Engineering in Medicine and Biology},
  year = {2023},
  doi = {10.1109/ojemb.2023.3248307},
}
Figure