Developing Clinical Artificial Intelligence for Obstetric Ultrasound to Improve Access in Underserved Regions: Protocol for a Computer-Assisted Low-Cost Point-of-Care UltraSound (CALOPUS) Study

Alice Self, Qingchao Chen, Bapu Koundinya Desiraju, Sumeet Dhariwal, Alexander D Gleed, Divyanshu Mishra, Ramachandran Thiruvengadam, Varun Chandramohan, Rachel Craik, Elizabeth Wilden, Ashok Khurana, Shinjini Bhatnagar, Aris T Papageorghiou, J Alison NobleJMIR Research Protocols 2022 · OpenReview

Abstract

Background: The World Health Organization recommends a package of pregnancy care that includes obstetric ultrasound scans. There are significant barriers to universal access to antenatal ultrasound, particularly because of the cost and need for maintenance of ultrasound equipment and a lack of trained personnel. As low-cost, handheld ultrasound devices have become widely available, the current roadblock is the global shortage of health care providers trained in obstetric scanning. Objective: The aim of this study is to improve pregnancy and risk assessment for women in underserved regions. Therefore, we are undertaking the Computer-Assisted Low-Cost Point-of-Care UltraSound (CALOPUS) project, bringing together experts in machine learning and clinical obstetric ultrasound. Methods: In this prospective study conducted in two clinical centers (United Kingdom and India), participating pregnant women were scanned and full-length ultrasounds were performed. Each woman underwent 2 consecutive ultrasound scans. The first was a series of simple, standardized ultrasound sweeps (the CALOPUS protocol), immediately followed by a routine, full clinical ultrasound examination that served as the comparator. We describe the development of a simple-to-use clinical protocol designed for nonexpert users to assess fetal viability, detect the presence of multiple pregnancies, evaluate placental location, assess amniotic fluid volume, determine fetal presentation, and perform basic fetal biometry. Results: Over 5571 scans have been acquired, from which 1,541,751 label annotations have been performed. An adapted protocol improved visualization of the cervix from 28% to 91% and classification of placental location from 82% to 94%. Conclusions: The CALOPUS study uses obstetric ultrasound videos and annotations to develop machine learning algorithms to address key clinical diagnostic questions pertaining to obstetric risk management.

Introduction

The World Health Organization recommends at least one ultrasound scan before 24 weeks of gestation to improve pregnancy outcomes. However, in low-resource settings, the primary barrier isn't just the cost of hardware—it's the critical shortage of trained personnel. Obstetric scanning is a highly skilled, operator-dependent task that takes years to master. While low-cost handheld ultrasound devices are now widely available, they remain "dumb" hardware without an expert to guide the probe and interpret the images. The CALOPUS (Computer-Assisted Low-Cost Point-of-Care UltraSound) study addresses this roadblock by creating a simplified clinical protocol and an AI pipeline capable of automated diagnosis.

Methods

The researchers established a prospective study across two clinical centers: the University of Oxford (UK) and the Translational Health Science and Technology Institute (India). The core innovation is a simplified 5-step "sweep" protocol. Instead of requiring a user to find specific anatomical planes (like the exact diameter of the fetal head), the user performs standardized, easy-to-learn sweeps across the maternal abdomen.

The CALOPUS simplified ultrasound scanning protocol and resulting AI annotations.

Each participant underwent two scans. First, the simplified CALOPUS sweeps (the AI input), followed immediately by a routine clinical ultrasound performed by an expert (the "ground truth" or comparator). The protocol was iteratively refined; for example, a specific U-shaped sweep along the pelvic brim, combined with a full maternal bladder, was found necessary to accurately visualize the cervix and placenta. To train the models, sonologists manually annotated over 1.5 million frames using bounding boxes for 11 anatomical features (fetal head, heart, spine, etc.) and placental location.

Results and Dataset

The study introduced a massive, newly curated dataset comprising 5,571 pregnant women recruited as of June 2022. This resulted in over 1.5 million label annotations across thousands of ultrasound videos. The dataset includes longitudinal data from 11 weeks' gestation through birth, specifically oversampling for "noncephalic" (breech) presentations to ensure the AI learns to recognize high-risk fetal positions.

Key results include:

  • Diagnostic Improvement: The adapted protocol improved the classification of placental location from 82% to 94%.
  • Visualization: Visualization of the cervix jumped from 28% to 91% when using the optimized sweep and a full bladder.
  • Reliability: The study achieved excellent inter-annotator agreement (Pooled $\kappa$ of 0.85 and bounding box overlap >93%), proving that the "ground truth" labels are highly consistent.

Discussion and Conclusion

Standard clinical protocols are designed for human eyes, not necessarily for machine learning. The CALOPUS researchers removed redundant steps to streamline data for AI while adding specific maneuvers that dramatically improved diagnostic accuracy. A practical finding was the "bladder effect": a well-filled bladder increased cervix visualization from 28% to 91%, critical for identifying conditions like placenta previa.

While the current data was collected on high-end machines by experts to ensure a "gold standard" dataset, the next phase involves testing how well these AI models perform when the sweeps are conducted by truly "non-expert" users on low-cost handheld devices. The ultimate success of CALOPUS hinges on whether the AI can maintain its accuracy when the probe is in the hands of a minimally trained community health worker. This study demonstrates that a combination of simplified clinical protocols and robust machine learning can potentially democratize prenatal care in underserved regions.

Summary updated

Abstract

Background: The World Health Organization recommends a package of pregnancy care that includes obstetric ultrasound scans. There are significant barriers to universal access to antenatal ultrasound, particularly because of the cost and need for maintenance of ultrasound equipment and a lack of trained personnel. As low-cost, handheld ultrasound devices have become widely available, the current roadblock is the global shortage of health care providers trained in obstetric scanning.Objective: The aim of this study is to improve pregnancy and risk assessment for women in underserved regions. Therefore, we are undertaking the Computer-Assisted Low-Cost Point-of-Care UltraSound (CALOPUS) project, bringing together experts in machine learning and clinical obstetric ultrasound.Methods: In this prospective study conducted in two clinical centers (United Kingdom and India), participating pregnant women were scanned and full-length ultrasounds were performed. Each woman underwent 2 consecutive ultrasound scans. The first was a series of simple, standardized ultrasound sweeps (the CALOPUS protocol), immediately followed by a routine, full clinical ultrasound examination that served as the comparator. We describe the development of a simple-to-use clinical protocol designed for nonexpert users to assess fetal viability, detect the presence of multiple pregnancies, evaluate placental location, assess amniotic fluid volume, determine fetal presentation, and perform basic fetal biometry. The CALOPUS protocol was designed using the smallest number of steps to minimize redundant information, while maximizing diagnostic information. Here, we describe how ultrasound videos and annotations are captured for machine learning.Results: Over 5571 scans have been acquired, from which 1,541,751 label annotations have been performed. An adapted protocol, including a low pelvic brim sweep and a well-filled maternal bladder, improved visualization of the cervix from 28% to 91% and classification of placental location from 82% to 94%. Excellent levels of intra- and interannotator agreement are achievable following training and standardization.Conclusions: The CALOPUS study is a unique study that uses obstetric ultrasound videos and annotations from pregnancies dated from 11 weeks and followed up until birth using novel ultrasound and annotation protocols. The data from this study are being used to develop and test several different machine learning algorithms to address key clinical diagnostic questions pertaining to obstetric risk management. We also highlight some of the challenges and potential solutions to interdisciplinary multinational imaging collaboration.International Registered Report Identifier (IRRID): RR1-10.2196/37374JMIR Res Protoc 2022;11(9):e37374doi:10.2196/37374
BibTeX
@inproceedings{self2022developing,
  author = {Alice Self and Qingchao Chen and Bapu Koundinya Desiraju and Sumeet Dhariwal and Alexander D Gleed and Divyanshu Mishra and Ramachandran Thiruvengadam and Varun Chandramohan and Rachel Craik and Elizabeth Wilden and Ashok Khurana and Shinjini Bhatnagar and Aris T Papageorghiou and J Alison Noble},
  title = {{Developing Clinical Artificial Intelligence for Obstetric Ultrasound to Improve Access in Underserved Regions: Protocol for a Computer-Assisted Low-Cost Point-of-Care UltraSound (CALOPUS) Study}},
  booktitle = {JMIR Research Protocols},
  year = {2022},
  url = {https://openreview.net/forum?id=eeA3cqe2U1},
}
Figure