An Assistive Role of a Machine Learning Network in Diagnosis of Middle Ear Diseases

Hayoung Byun, Sangjoon Yu, Jaehoon Oh, Junwon Bae, Myeong Seong Yoon, Seung Hwan Lee, Jae Ho Chung, Tae Hyun KimJournal of Clinical Medicine 2021 · DOI

TL;DR

This paper introduces a novel machine learning network, "ResNet18 + Shuffle," that achieves high accuracy (97.18%) in diagnosing middle ear diseases from tympanic membrane images. Crucially, the study demonstrates that this AI tool acts as a valuable assistant, significantly improving the diagnostic accuracy of junior otolaryngology residents.

What Problem Does This Paper Address?

Diagnosing middle ear diseases, particularly otitis media, is a common but sometimes challenging task. While tympanic membrane evaluation is the first step, it can be difficult, especially for non-specialists or in resource-limited settings. Existing AI approaches have shown promise in image classification for medical conditions, but there's a gap in understanding their assistive or educational role for clinicians, particularly for junior residents who are still developing their diagnostic skills. This paper aims to bridge that gap by developing an AI diagnostic tool and evaluating its impact on resident physician performance.

Methods

The core of this research is the development of a custom deep learning architecture. They started with a ResNet18 backbone, a well-established convolutional neural network for image classification. The novelty lies in the integration of a "Shuffle Attention Module." This module is designed to generate 3D attention maps, capturing both spatial and channel information within the feature maps. This is achieved through a "space to depth" operation, which effectively reorganizes feature dimensions to enhance representation power with minimal additional parameters. This attention mechanism is hypothesized to improve the network's ability to focus on critical diagnostic features within the tympanic membrane images. For validation, they employed k-fold cross-validation and Grad-CAM for visual interpretability, allowing them to see which image regions the network focused on.

Datasets

The study utilized a dataset of 2272 tympanic membrane images collected from patients who visited an otolaryngology department between January 2015 and December 2018. These images were captured using a Digital Videoscope and were meticulously labeled by three otolaryngology specialists into four categories: normal, otitis media with effusion (OME), chronic otitis media (COM), and cholesteatoma. For final testing and evaluation of the assistive role, a subset of 71 representative cases, carefully selected by senior otolaryngologists, was used.

Key Insights and Ideas

  • AI as a Diagnostic Co-pilot: The most significant insight is that AI doesn't just replace human diagnosticians; it can significantly augment their capabilities. The study clearly shows that even with a highly accurate AI, human oversight and interpretation remain crucial, and the AI can guide them to better decisions.
  • Attention is Key: The introduction of the "Shuffle Attention Module" highlights the growing importance of attention mechanisms in deep learning for medical imaging. By allowing the network to dynamically focus on relevant image regions, it can achieve higher accuracy and potentially better interpretability.
  • Bridging the Skill Gap: The research directly addresses the challenge of training junior clinicians. By providing an AI assistant, the study suggests a pathway to accelerate learning and improve diagnostic consistency among less experienced physicians.
  • Subtle Differentiations: The Grad-CAM visualizations reveal how the AI differentiates between conditions. For instance, it correctly identifies the pars tensa for normal cases but focuses on perforations for COM and retraction pockets for cholesteatoma, showcasing its ability to pick up subtle visual cues.

Key Results

  • High Network Accuracy: The developed "ResNet18 + Shuffle" network achieved an impressive diagnostic accuracy of 97.18% on a test set of 71 representative cases.
  • Robust Validation: Five-fold cross-validation demonstrated a consistent accuracy of over 93%.
  • Significant Improvement for Residents: When assisted by the AI, the diagnostic accuracy of junior residents increased from an average of 82.9% to 90.0%, a gain of 7.1% (ranging from 1.4% to 18.4% individually).
  • AI vs. Human Errors: While the AI made errors in distinguishing OME from COM and in some cholesteatoma cases, it consistently outperformed residents in differentiating normal tympanic membranes from OME. Residents, however, were better at diagnosing attic cholesteatoma in specific challenging cases.

Limitations and Future Work

  • Limited Disease Scope: The study focused on four specific middle ear conditions. The authors acknowledge that many other ear diseases, including acute otitis media, chronic conditions with low prevalence, and external auditory canal malignancies, were not included.
  • Single-Institution Data: The dataset was collected and preprocessed at a single institution. For broader applicability, external validation with data from multiple institutions and diverse imaging equipment is necessary.
  • Data Scarcity for Rare Cases: The AI struggled with atypical cases of OME and cholesteatoma, likely due to their limited representation in the training data.
  • Future Directions: The authors suggest expanding the network to include a wider range of ear diseases and conducting external validation. They also emphasize the potential for AI to be integrated into medical education and practice, fostering a complementary relationship between human expertise and artificial intelligence.

Summary updated

Abstract

The present study aimed to develop a machine learning network to diagnose middle ear diseases with tympanic membrane images and to identify its assistive role in the diagnostic process. The medical records of subjects who underwent ear endoscopy tests were reviewed. From these records, 2272 diagnostic tympanic membranes images were appropriately labeled as normal, otitis media with effusion (OME), chronic otitis media (COM), or cholesteatoma and were used for training. We developed the "ResNet18 + Shuffle" network and validated the model performance. Seventy-one representative cases were selected to test the final accuracy of the network and resident physicians. We asked 10 resident physicians to make diagnoses from tympanic membrane images with and without the help of the machine learning network, and the change of the diagnostic performance of resident physicians with the aid of the answers from the machine learning network was assessed. The devised network showed a highest accuracy of 97.18%. A five-fold validation showed that the network successfully diagnosed ear diseases with an accuracy greater than 93%. All resident physicians were able to diagnose middle ear diseases more accurately with the help of the machine learning network. The increase in diagnostic accuracy was up to 18% (1.4% to 18.4%). The machine learning network successfully classified middle ear diseases and was assistive to clinicians in the interpretation of tympanic membrane images.
BibTeX
@article{byun2021assistive,
  author = {Hayoung Byun and Sangjoon Yu and Jaehoon Oh and Junwon Bae and Myeong Seong Yoon and Seung Hwan Lee and Jae Ho Chung and Tae Hyun Kim},
  title = {{An Assistive Role of a Machine Learning Network in Diagnosis of Middle Ear Diseases}},
  journal = {Journal of Clinical Medicine},
  year = {2021},
  doi = {10.3390/jcm10153198},
}
Figure