Real-World Verification of Artificial Intelligence Algorithm-Assisted Auscultation of Breath Sounds in Children

Jing Zhang, Han-song Wang, Hongyuan Zhou, Bin Dong, Lei Zhang, Fen Zhang, Shijian Liu, Yufen Wu, Shuhua Yuan, Mingyu Tang, Wenfang Dong, Lin Jie, Ming Chen, Xing Tong, Liebin Zhao, Yong YinFrontiers in Pediatrics 2021 · DOI

Abstract

Objective: Lung auscultation is critical for diagnosing pediatric pulmonary diseases. This study evaluates an artificial intelligence (AI) algorithm's performance in detecting breath sounds within a real-world clinical environment.

Method: Breath sounds from 112 children at Shanghai Children's Medical Center (SCMC) were recorded using an electronic stethoscope. A Gold Standard (GS) was established by expert consensus. The AI algorithm's performance was compared against six general pediatricians across various patient ages and recording locations.

Result: Of 672 recordings, 627 were analyzed. The AI algorithm achieved 77.7% accuracy, significantly higher than the 59.9% achieved by general pediatricians ($p < 0.001$). AI sensitivity and F1-scores for crackles and wheezes also surpassed human performance. Accuracy was highest in infants under 12 months (81.3%).

Conclusion: AI-assisted auscultation via electronic stethoscopes outperforms general pediatricians in identifying adventitious breath sounds, offering a robust tool for clinical and remote screening.

Introduction

Stethoscope auscultation remains a cornerstone of pediatric medicine, yet its effectiveness is often limited by the subjective nature of human hearing and varying levels of clinical experience. Distinguishing between normal breath sounds and adventitious sounds—such as crackles or wheezes—requires significant expertise, creating a diagnostic gap between specialized pulmonologists and general practitioners. Furthermore, traditional acoustic stethoscopes lack the capability for data storage or objective longitudinal monitoring.

This study investigates whether an AI-integrated electronic stethoscope can bridge this expertise gap. By utilizing machine learning to analyze recordings in noisy, real-world clinical settings, researchers aimed to provide an objective verification tool for pediatric respiratory health.

Methods

The study implemented a "record-upload-analyze" workflow to evaluate the AI's diagnostic capabilities.

Data Collection and Hardware

Nurses recorded 9-second clips from six specific chest locations on 112 hospitalized children at SCMC. The hardware setup included the Yunting model II electronic stethoscope connected to a mobile device for data transmission.

The electronic stethoscope setup used for recording and transmitting breath sounds.

Auscultation Protocol

Six standardized locations on the chest and back were used to ensure comprehensive coverage of the lung fields. This protocol allowed for a systematic assessment of the respiratory system across different patient demographics.

The six standardized chest and back locations used for breath sound collection.

The AI Pipeline

The AI system processed the audio through several stages:

  1. Pre-processing: Filtering ambient noise and identifying temporal characteristics (e.g., crackles as discontinuous signals <20ms).
  2. Feature Extraction: Utilizing Wavelet Packet Decomposition to extract spectral features from non-stationary breath signals.
  3. Classification: A Support Vector Machine (SVM), previously trained on over 12,000 real-world recordings, classified the sounds as normal, crackle, or wheeze.

Evaluation Baseline

A "Gold Standard" (GS) was established through the consensus of eight expert pulmonologists. To provide a clinical baseline, six general pediatricians also evaluated the recordings independently.

Results

A total of 627 high-quality recordings were analyzed, comprising crackles (23.1%), wheezes (38.4%), and normal sounds (38.4%).

AI vs. Human Performance

The AI algorithm demonstrated superior diagnostic accuracy across all primary metrics:

  • Overall Accuracy: AI (77.7%) vs. General Pediatricians (59.9%), $p < 0.001$.
  • Crackle Detection: The AI's F1-score was 80.9%, nearly doubling the pediatricians' score of 42.7%.
  • Wheeze Detection: The AI achieved an F1-score of 80.9%, compared to 72.5% for the pediatricians.

Age and Location Factors

The AI's performance was notably robust in infants under 12 months, reaching an accuracy of 81.3%. This is likely due to thinner chest walls in younger patients providing clearer acoustic signals. Importantly, the algorithm maintained consistent accuracy across all six recording locations, suggesting that simplified screening protocols may be feasible.

Discussion

The findings highlight AI's potential as an "expertise leveler." By outperforming general pediatricians, the algorithm can elevate the diagnostic quality in primary care settings to a level closer to specialized centers. The system's resilience to "real-world" noise—including ambient ward sounds and patient movement—indicates its readiness for clinical integration.

Limitations

Despite its success, the algorithm currently cannot recognize "mixed" sounds (simultaneous crackles and wheezes). Additionally, the sample size for older children (>60 months) was relatively small, and the study was conducted at a single center.

Conclusion

AI-assisted auscultation provides a more accurate and objective assessment of pediatric breath sounds than general clinical evaluation. Integrating this technology into community health centers and telemedicine platforms could significantly improve respiratory screening and chronic disease management for children, especially in resource-limited areas.

Summary updated

Abstract

<b>Objective:</b> Lung auscultation plays an important role in the diagnosis of pulmonary diseases in children. The objective of this study was to evaluate the use of an artificial intelligence (AI) algorithm for the detection of breath sounds in a real clinical environment among children with pulmonary diseases. <b>Method:</b> The auscultations of breath sounds were collected in the respiratory department of Shanghai Children's Medical Center (SCMC) by using an electronic stethoscope. The discrimination results for all chest locations with respect to a gold standard (GS) established by 2 experienced pediatric pulmonologists from SCMC and 6 general pediatricians were recorded. The accuracy, sensitivity, specificity, precision, and F1-score of the AI algorithm and general pediatricians with respect to the GS were evaluated. Meanwhile, the performance of the AI algorithm for different patient ages and recording locations was evaluated. <b>Result:</b> A total of 112 hospitalized children with pulmonary diseases were recruited for the study from May to December 2019. A total of 672 breath sounds were collected, and 627 (93.3%) breath sounds, including 159 crackles (23.1%), 264 wheeze (38.4%), and 264 normal breath sounds (38.4%), were fully analyzed by the AI algorithm. The accuracy of the detection of adventitious breath sounds by the AI algorithm and general pediatricians with respect to the GS were 77.7% and 59.9% (<i>p</i> < 0.001), respectively. The sensitivity, specificity, and F1-score in the detection of crackles and wheeze from the AI algorithm were higher than those from the general pediatricians (crackles 81.1 vs. 47.8%, 94.1 vs. 77.1%, and 80.9 vs. 42.74%, respectively; wheeze 86.4 vs. 82.2%, 83.0 vs. 72.1%, and 80.9 vs. 72.5%, respectively; <i>p</i> < 0.001). Performance varied according to the age of the patient, with patients younger than 12 months yielding the highest accuracy (81.3%, <i>p</i> < 0.001) among the age groups. <b>Conclusion:</b> In a real clinical environment, children's breath sounds were collected and transmitted remotely by an electronic stethoscope; these breath sounds could be recognized by both pediatricians and an AI algorithm. The ability of the AI algorithm to analyze adventitious breath sounds was better than that of the general pediatricians.
BibTeX
@inproceedings{zhang2021real,
  author = {Jing Zhang and Han-song Wang and Hongyuan Zhou and Bin Dong and Lei Zhang and Fen Zhang and Shijian Liu and Yufen Wu and Shuhua Yuan and Mingyu Tang and Wenfang Dong and Lin Jie and Ming Chen and Xing Tong and Liebin Zhao and Yong Yin},
  title = {{Real-World Verification of Artificial Intelligence Algorithm-Assisted Auscultation of Breath Sounds in Children}},
  booktitle = {Frontiers in Pediatrics},
  year = {2021},
  doi = {10.3389/fped.2021.627337},
}
Figure