AutoAudio: Deep Learning for Automatic Audiogram Interpretation

Matthew G. Crowson, Jong Wook Lee, Amr Hamour, Rafid Mahmood, Aaron Babier, Vincent Lin, Debara L. Tucci, Timothy C. Y. Chan

TL;DR

Researchers have developed "AutoAudio," a deep learning proof-of-concept that automatically interprets audiogram plots with up to 97.5% accuracy. By treating audiograms as images rather than just data points, the model can classify hearing loss types, potentially bridging the gap in regions with severe shortages of trained audiologists.

What Problem Does This Paper Address?

Hearing loss is a leading cause of disability worldwide, yet there is a massive global deficit in audiology workforce capacity. In many developing nations, there is less than one audiologist per million people. Even in developed countries like the U.S., demand is forecasted to outpace supply by 50% in the near future.

While portable diagnostic tools have made testing easier, the interpretation of those tests still requires an expert. Most existing automated solutions focus on digital data points, but a vast amount of historical and current clinical data exists in the form of traditional "plot-based" audiograms—graphs showing hearing thresholds across frequencies. This paper tackles the challenge of automating the interpretation of these visual records to triage patients and digitize archives rapidly.

Methods

The authors developed AutoAudio using a computer vision approach. Instead of feeding the model raw numbers, they treated audiogram reports as images.

  • Architecture: They employed Transfer Learning, taking four variants of the Residual Network (ResNet) architecture (ResNet-32, 50, 101, and 152) originally trained on the massive ImageNet database.
  • Training: Because audiograms look nothing like the natural objects in ImageNet, they re-trained every layer of the networks. They used image augmentation (rotation, warping, contrast adjustments) to artificially expand their training set, simulating real-world variations like scanned pages being slightly crooked.
  • Interpretability: To understand how the model made decisions, they used Grad-CAM (Gradient-weighted Class Activation Mapping). This creates heat-maps over the audiogram images, highlighting which visual regions (like the gap between air and bone conduction lines) most influenced the model’s prediction.

Datasets

The study used a convenience sample of 1,007 audiogram reports from adult patients at the Sunnybrook Health Sciences Center (2017–2019).

  • Labels: Each image was labeled by expert audiologists into one of four categories: normal, conductive loss, sensorineural loss, or mixed loss.
  • Split: The data was split 80/20 into a training set (n=806) and a "hold-out" validation set (n=201) to test the model on unseen data.
  • Composition: The corpus included 435 sensorineural, 214 mixed, 207 normal, and 151 conductive hearing loss cases.

Key Insights and Ideas

  • Visual Logic vs. Rule-Based Logic: Traditional automation uses "if-then" rules (e.g., "if the gap is >15dB, then it's conductive"). AutoAudio, however, "learns" these spatial relationships independently. The Grad-CAM heat-maps confirmed that the model naturally focused on the "air-bone gap," effectively rediscovering clinical principles through pure pattern recognition.
  • The "Mixed Loss" Challenge: The model struggled most with "mixed" hearing loss. This makes sense clinically: mixed loss requires the simultaneous observation of two different patterns (thresholds below normal and a gap between air/bone conduction), making it the most conceptually complex morphology to identify visually.
  • Diminishing Returns on Depth: Interestingly, deeper networks didn't always mean better results. While ResNet-101 was the winner, the performance difference between the 32-layer and 152-layer versions was within a narrow 2% margin. For clinical deployment, a "lighter" model that trains faster might be preferable.

Key Results

  • Top Accuracy: The ResNet-101 architecture achieved the highest classification accuracy at 97.5%.
  • Efficiency: Training times were relatively short, ranging from 2 to 7 hours depending on the network depth.
  • Reliability: The model excelled at identifying "normal" plots. Most errors occurred when the model failed to recognize a bone conduction threshold that was only slightly below the normal range, leading it to misclassify a mixed loss as a purely conductive one.

Limitations and Future Work

The authors identify several hurdles before AutoAudio hits the clinic:

  • Generalizability: The current model was trained on standardized, hand-written plots from a single institution. It might struggle with different software outputs, different scales, or plots that combine both ears on one graph.
  • Clinical Context: The model interprets morphology alone. It does not yet account for patient history, tympanometry, or word recognition scores, which are vital for a full medical diagnosis.
  • Head-to-Head Testing: The next step involves a direct "man vs. machine" comparison, pitting AutoAudio against established audiologists and trainees to assess inter-rater reliability.

Ultimately, the authors see AutoAudio as a "backwards compatible" tool—able to digitize decades of paper records while providing instant triaging in resource-limited settings.

Summary updated

Abstract

Hearing loss is the leading human sensory system loss, and one of the leading causes for years lived with disability with significant effects on quality of life, social isolation, and overall health. Coupled with a forecast of increased hearing loss burden worldwide, national and international health organizations have urgently recommended that access to hearing evaluation be expanded to meet demand. The objective of this study was to develop ‘AutoAudio’ – a novel deep learning proof-of-concept model that accurately and quickly interprets diagnostic audiograms. Adult audiogram reports representing normal, conductive, mixed and sensorineural morphologies were used to train different neural network architectures. Image augmentation techniques were used to increase the training image set size. Classification accuracy on a separate test set was used to assess model performance. The architecture with the highest out-of-training set accuracy was ResNet-101 at 97.5%. Neural network training time varied between 2 to 7 h depending on the depth of the neural network architecture. Each neural network architecture produced misclassifications that arose from failures of the model to correctly label the audiogram with the appropriate hearing loss type. The most commonly misclassified hearing loss type were mixed losses. Re-engineering the process of hearing testing with a machine learning innovation may help enhance access to the growing worldwide population that is expected to require audiologist services. Our results suggest that deep learning may be a transformative technology that enables automatic and accurate audiogram interpretation.
BibTeX
@article{crowson2020autoaudio,
  author = {Matthew G. Crowson and Jong Wook Lee and Amr Hamour and Rafid Mahmood and Aaron Babier and Vincent Lin and Debara L. Tucci and Timothy C. Y. Chan},
  title = {{AutoAudio: Deep Learning for Automatic Audiogram Interpretation}},
  journal = {J. Medical Syst.},
  year = {2020},
  doi = {10.1007/s10916-020-01627-1},
}

Loading related content...

Figure