Multi-center Integrating Radiomics, Structured Reports, and Machine Learning Algorithms for Assisted Classification of COVID-19 in Lung Computed Tomography

Marcos Antônio Dórea Machado, Ronnyldo R. E. Silva, Mauro Namías, Andreia S. Lessa, Margarida Célia Lima Costa Neves, Carolina T. A. Silva, Danillo Menezes Oliveira, Thamiris Rosado Reina, Arquimedes A. B. Lira, Leandro M. Almeida, Cleber Zanchettin, Eduardo Martins NettoJournal of Medical and Biological Engineering 2023 · DOI

Introduction

Distinguishing COVID-19 from other types of viral or bacterial pneumonia on Computed Tomography (CT) scans is a significant clinical challenge. While certain visual patterns are typical of COVID-19, they are often not pathognomonic, leading to diagnostic ambiguity and high inter-rater variability among clinicians. This study investigates whether a "human-in-the-loop" approach—integrating standardized structured reports, quantitative radiomic features, and machine learning (ML) algorithms—can enhance diagnostic accuracy and consistency.

Methods

The researchers implemented a multi-center pipeline across three Brazilian academic hospitals, utilizing a dataset of 128 subjects (64 COVID-19 cases and 64 pre-pandemic pneumonia controls). The data was partitioned into a training cohort (n=73) and an independent validation cohort (n=55).

The methodology employed two primary data streams:

  1. Structured Reports: Radiologists completed a 10-item binary checklist regarding lesion morphology and distribution. These qualitative observations were processed using an XgBoost algorithm.
  2. Radiomics: Lung lesions were automatically segmented using a 2D convolutional neural network. From these regions of interest (ROIs), 93 quantitative features (texture, shape, and intensity) were extracted. To ensure robustness, features were filtered based on their reproducibility (Intraclass Correlation Coefficient > 0.90) across various image perturbations, such as erosion and different kernels.
Radiomics pipeline for feature extraction and selection.

The final evaluation compared the performance of two experienced physicians (a radiologist and a pneumologist) working independently versus their performance when assisted by the ML models 30 days later.

Visual representation of lung CT segmentation and image processing filters used for radiomic analysis.

Results and Key Insights

The integration of ML significantly improved diagnostic performance, particularly in terms of specificity. While humans are generally sensitive to detecting pneumonia, they often struggle with specificity—the ability to correctly identify the specific cause.

  • Diagnostic Accuracy: Physicians alone achieved a mean sensitivity of 83.4% and a specificity of 64.3%. With AI support, sensitivity rose to 87.1%, and specificity increased dramatically to 91.1%.
  • Model Performance: On the validation set, the Radiomics model and the Structured Report model achieved AUCs of 0.84 and 0.82, respectively. The radiologist, when assisted by AI, reached a peak specificity of 96.4% and an AUC of 0.97.
  • Inter-rater Reliability: A critical finding was the improvement in Cohen’s Kappa coefficient. ML assistance moved the agreement between the radiologist and the pneumologist from "moderate" to "substantial," suggesting that AI can help harmonize diagnostic standards across different medical specialties.

Discussion and Limitations

The study highlights that standardizing clinical descriptions through structured reports makes qualitative data "minable" for AI, acting as a powerful "specificity filter." However, several limitations remain. The sample size of 128 subjects is relatively small, resulting in wide confidence intervals. Additionally, the models were specifically trained to distinguish COVID-19 from other pneumonias and may not generalize to other lung pathologies like cancer. Future research will focus on fusing the radiomics and structured report models into a single unified score and validating the system in larger, prospective clinical trials.

Summary updated

BibTeX
@article{machado2023multi,
  author = {Marcos Antônio Dórea Machado and Ronnyldo R. E. Silva and Mauro Namías and Andreia S. Lessa and Margarida Célia Lima Costa Neves and Carolina T. A. Silva and Danillo Menezes Oliveira and Thamiris Rosado Reina and Arquimedes A. B. Lira and Leandro M. Almeida and Cleber Zanchettin and Eduardo Martins Netto},
  title = {{Multi-center Integrating Radiomics, Structured Reports, and Machine Learning Algorithms for Assisted Classification of COVID-19 in Lung Computed Tomography}},
  journal = {Journal of Medical and Biological Engineering},
  year = {2023},
  doi = {10.1007/s40846-023-00781-4},
}
Figure