Maternal early warning scores shown to be methodologically weak and at high risk of bias

Mae Chester‐Jones, Shona Kirtley, Fema Er, Vidoushee Jogarah, Yu Qiao, R. Tunn, Naomi Vides, Peter Watkinson, Marian Knight, Stephen Gerry, Gary S. CollinsJournal of Clinical Epidemiology 2025 · DOI

TL;DR

This systematic review critically appraises the development and validation of maternal early warning scores (MOEWSs), finding that most are methodologically weak and at high risk of bias. This suggests that current MOEWSs may perform poorly in clinical practice and could even be harmful, highlighting an urgent need for more robustly developed and validated tools.

What Problem Does This Paper Address?

Maternal deaths and life-threatening complications during pregnancy and postpartum are significant public health concerns. Modified Obstetric Early Warning Scores (MOEWSs) are clinical prediction models (CPMs) designed to identify early signs of deterioration in pregnant and postpartum women, aiming to prevent severe outcomes. Despite their widespread use in clinical practice, there's a lack of standardization and a concern about the quality of their development and validation. This paper addresses the critical gap in understanding the methodological rigor of existing MOEWSs, questioning whether their current design and validation are sufficient to ensure reliable and safe clinical application.

Methods

The researchers conducted a systematic review of studies describing the development or external validation of MOEWSs. They searched multiple databases (Medline, CINAHL, EMBASE, Web of Science) for studies published between January 1, 2000, and December 31, 2022. To critically appraise the included studies, they employed two key tools: the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modeling Studies (CHARMS) checklist and the Prediction Model Risk of Bias Assessment Tool (PROBAST). The review focused on assessing the methodological conduct, completeness of reporting, and risk of bias in these studies.

Datasets

The study did not introduce new datasets. Instead, it analyzed existing studies that developed or validated MOEWSs. The data for the review were extracted from 18,874 identified studies, with 20 ultimately included in the analysis. These included studies utilized various data sources, including retrospective cohorts, prospective cohorts, and simulated datasets, with sample sizes ranging from 123 to over 176,000 pregnant women.

Key Insights and Ideas

  • Widespread use, poor foundation: MOEWSs are commonly used in clinical practice, yet the vast majority are not developed using recommended statistical methods. Many rely on clinical consensus rather than data-driven approaches.
  • Data-driven models fall short: Even when data-driven approaches were used, studies often failed to address crucial statistical challenges like repeated measures and missing data, and they rarely reported adequate performance metrics.
  • High risk of bias is the norm: Nearly all development and validation studies were rated at high risk of bias. This is attributed to poor reporting, inadequate handling of missing data, and a lack of comprehensive model evaluation.
  • Validation is often superficial: External validation studies frequently used historical data collected for other purposes and often validated scores developed through clinical consensus, rather than more robust statistical models.
  • Underrepresentation and missed opportunities: Demographic characteristics, particularly those associated with higher risk (e.g., ethnicity), were poorly reported, making it unclear how MOEWSs perform in diverse populations. Open science practices were also rarely followed, hindering reproducibility.

Key Results

  • Out of 18,874 identified studies, 20 met the inclusion criteria.
  • 10 studies were model development studies (5 with external validation), and 10 were validation-only studies.
  • Only four development studies used statistical methods; the remaining six used clinical consensus.
  • 95% of development studies and 73% of validation studies were rated at high risk of bias.
  • None of the data-driven models were independently validated.
  • Key statistical challenges like handling missing data and repeated measures were rarely addressed.

Limitations and Future Work

The authors acknowledge that previous reviews have also identified poor quality in MOEWS models. A limitation is that many MOEWS charts in use have not been published in peer-reviewed articles, making their development details unclear.

The paper strongly recommends future work to:

  • Develop MOEWSs using robust statistical modeling: This includes utilizing the repeated measures nature of the data and employing appropriate statistical methods.
  • Improve data description: Future studies should report key demographic characteristics, especially those linked to adverse outcomes, and assess model performance across different stages of pregnancy.
  • Address missing data comprehensively: Methods for handling missing data should be considered at all stages of model development, validation, and implementation, avoiding inefficient methods like complete case analysis.
  • Embrace open science: Adopting open science practices would enhance reproducibility and reduce research waste.
  • Focus on diverse populations: Investigate how MOEWSs perform in underrepresented groups and those most at risk of morbidity and mortality.

Summary updated

Abstract

There is a lack of MOEWSs developed using methods that follow recommended statistical guidelines. Substantial problems with the methodological quality of included development and validation studies, along with high risk of bias,indicating published scores could perform poorly or be potentially harmful if used in clinical practice. Future work should address handling missing data and repeated measures and consider how an MOEWS will perform in different populations and key subgroups.
BibTeX
@article{chesterjones2025maternal,
  author = {Mae Chester‐Jones and Shona Kirtley and Fema Er and Vidoushee Jogarah and Yu Qiao and R. Tunn and Naomi Vides and Peter Watkinson and Marian Knight and Stephen Gerry and Gary S. Collins},
  title = {{Maternal early warning scores shown to be methodologically weak and at high risk of bias}},
  journal = {Journal of Clinical Epidemiology},
  year = {2025},
  doi = {10.1016/j.jclinepi.2025.111833},
}
Figure