Abstract
With interest, we read the article by Banoei et al. on machine learning (ML) models to predict mortality among COVID-19 patients. They refer to other studies that failed to predict mortality using ‘conventional statistical analysis’, after which they present a linear ML model as a better-suited method for such complex medical problems. We feel such a claim creates an image around ML as an alternative technique that offers solutions where statistical modeling fails. However, ML and statistical modeling are tightly interwoven.
Introduction: The "Hype" Gap in Clinical AI
In the urgent effort to apply artificial intelligence to the COVID-19 pandemic, researchers have frequently framed machine learning (ML) as a "silver bullet" capable of succeeding where traditional statistical analysis falls short. This perspective often creates conceptual friction between clinicians and data scientists. Clinicians may view ML as a "black box" that magically outperforms traditional models, leading to unrealistic expectations or a lack of transparency in how models actually function.
The core challenge addressed in this discussion is twofold: the perceived gap between ML and statistics, and the need for methodological rigor. There is a significant lack of consensus on how to differentiate ML from multivariate statistics, which often leads to inconsistent reporting and makes it difficult to judge the clinical relevance of new predictive models.
Methodological Debate: PLS and SIMPLS
The debate centers on the classification of specific methods like Partial Least Squares (PLS) and its variant, SIMPLS.
Smit et al. argue that these methods are fundamentally forms of statistical analysis. They point to systematic reviews indicating no significant performance difference between ML and standard logistic regression for clinical prediction tasks. In contrast, Banoei et al. defend the categorization of SIMPLS as an ML-based method, highlighting its specific advantages in handling multicollinearity—where variables are highly correlated—and its lower risk of overfitting compared to more complex architectures like Random Forests (RF), Support Vector Machines (SVM), or Artificial Neural Networks (ANN).
Key Insights: The Semantic Blur and Reporting Standards
A primary takeaway from this exchange is that the line between "Machine Learning" and "Multivariate Analysis" is increasingly thin. While the terms are often used interchangeably, the distinction matters less than the quality of the model's development and reporting.
- Reporting is Paramount: Adherence to standards like TRIPOD (Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis) is essential. Without clear documentation on how data was aggregated (e.g., the frequency and timing of SpO2 measurements), a model's clinical utility remains questionable.
- Interpretability vs. Complexity: The value of ML in an Intensive Care Unit (ICU) setting is not just "automatic learning," but the ability to identify the most differentiating variables in a way that is functional for researchers facing a pandemic.
- Intended Use: A model’s development must be driven by its intended clinical application, such as resource allocation versus individual prognosis. If the intended use is ambiguous, the underlying methodological choices may be flawed.
Clinical Utility and Results
As this is a correspondence piece, the "results" are qualitative, reflecting a growing consensus on the future of clinical modeling:
- No Universal Superiority: There is no single ML method that is inherently superior to all others across all clinical contexts.
- Performance Parity: ML does not consistently outperform "conventional" statistics in discriminative performance (e.g., AUC/ROC). However, ML may offer superior tools for managing large, high-dimensional, and "messy" clinical datasets.
- Methodology over Hype: The success of ML in the modern ICU depends on rigorous methodology and clear communication of limitations to the clinicians who will use these tools.
Limitations and Future Directions
The discussion highlights several persistent risks in the field. Incomplete reporting, such as failing to specify how frequently measured predictors are aggregated, can lead to erroneous conclusions. Furthermore, describing ML in overly technical or "extraneous" terms risks alienating clinicians.
Moving forward, both groups advocate for developing models with an explicit intended use in mind. The goal should be a unified approach to reporting findings, ensuring that clinical relevance is judged fairly and transparently, regardless of whether a model is labeled as "statistical" or "machine learning."
Summary updated