Performance of an artificial intelligence algorithm for interpreting lung sounds from children hospitalised with pneumonia in Malawi

Abstract

Background: Pneumonia is a leading cause of death in under five year olds globally. World Health Organization (WHO) pneumonia diagnostic guidelines rely on non-specific clinical findings. Lung auscultation could improve pneumonia diagnosis, but conventional stethoscopes have implementation challenges. To address this, we developed an artificial intelligence (AI)-enabled digital auscultation system. We evaluated the system's AI lung sound analysis algorithm in children with severe pneumonia in Malawi. Methods: We enrolled children aged 2-59 months hospitalised with WHO-defined severe pneumonia. A study physician recorded lung sounds with a digital stethoscope at six chest positions. Recordings were de-identified, filtered, and interpreted by a trained and certified physician listening panel. Interpretable recordings were analysed by the AI algorithm. We evaluated the agreement of normal (absence of adventitial lung sounds) vs. abnormal (presence of adventitial lung sounds) classifications, by chest position and by patient, between the AI algorithm and the listening panel using raw percent agreement kappa statistics, both unadjusted and adjusted for chance agreement. Results: We enrolled 100 children and analysed 95 with interpretable recordings. The median age was 12.6 months (interquartile range (IQR) = 5.4, 19.0) and 54% (51 / 95) were female. Among interpretable recordings, 59.2% (294 / 497) of chest positions were abnormal per the listening panel compared to 52.7% (262 / 497) per the AI algorithm. The listening panel and AI algorithm agreed on classifications in 83.1% (413 / 497) of chest positions (unadjusted kappa 0.7; adjusted kappa 0.7) and 91.6% (87/95) of patients (unadjusted kappa 0.7; adjusted kappa 0.8). The AI algorithm's sensitivity and specificity for identifying abnormal lung sounds, compared to the listening panel, were 80.3% and 87.2% for chest positions and 96.3%, and 66.7% for patients. Conclusions: This AI lung sound classification algorithm accurately identified abnormal lung sounds in children with severe pneumonia. Next steps include training the algorithm to identify uninterpretable recordings and different abnormal sounds.