Multimodal Deep Learning for ARDS Detection

Stefan Broecker, Jason Y. Adams, Girish Kumar, Rachael A. Callcut, Yuan Ni, Thomas StrohmermedRxiv 2025 · DOI

Abstract

Objective: Poor outcomes in acute respiratory distress syndrome (ARDS) can be alleviated with tools that support early diagnosis. Current machine learning methods for detecting ARDS do not take full advantage of the multimodality of ARDS pathophysiology. We developed a multimodal deep learning model that uses imaging data, continuously collected ventilation data, and tabular data derived from a patient’s electronic health record (EHR) to make ARDS predictions.

Materials and Methods: A chest radiograph (x-ray), at least two hours of ventilator waveform (VWD) data within the first 24 hours of intubation, and EHR-derived tabular data were used from 220 patients admitted to the ICU to train a deep learning model. The model uses pretrained encoders for the x-rays and ventilation data and trains a feature extractor on tabular data. Encoded features for a patient are combined to make a single ARDS prediction. Ablation studies for each modality assessed their effect on the model’s predictive capability.

Results: The trimodal model achieved an area under the receiver operator curve (AUROC) of 0.86 with a 95% confidence interval of 0.01. This was a statistically significant improvement (p<0.05) over single modality models and bimodal models trained on VWD+tabular and VWD+x-ray data.

Discussion and Conclusion: Our results demonstrate the potential utility of using deep learning to address complex conditions with heterogeneous data. More work is needed to determine the additive effect of modalities on ARDS detection. Our framework can serve as a blueprint for building performant multimodal deep learning models for conditions with small, heterogeneous datasets.

Introduction

Acute Respiratory Distress Syndrome (ARDS) is a life-threatening lung condition affecting approximately 10% of ICU patients, with mortality rates nearing 40%. Early diagnosis is critical for survival, yet it remains notoriously difficult because ARDS is inherently multimodal. Its diagnosis requires clinicians to synthesize information from chest imaging, physiological signals (such as oxygenation and ventilator settings), and clinical history.

While machine learning has shown promise in medical diagnostics, most existing models for ARDS are unimodal, focusing on a single data type. Furthermore, deep learning typically requires massive datasets that are difficult to aggregate in specialized clinical contexts. This research addresses these hurdles by developing a trimodal deep learning framework that effectively fuses heterogeneous data streams and leverages transfer learning to perform well on relatively small patient cohorts.

Methods

The researchers proposed a trimodal architecture that processes three distinct data types: chest X-rays (CXR), continuous ventilator waveform data (VWD) representing airflow at 50Hz, and EHR-derived tabular data (labs, vitals, and demographics).

The trimodal architecture utilizes pretrained encoders for X-ray and VWD data, alongside a tabular feature extractor, to generate a unified ARDS prediction.

A key innovation of this approach is the use of pretrained encoders to mitigate data scarcity. For imaging, a ResNet-50 backbone pretrained on radiological tasks was used. For ventilator waveforms, the team employed transfer learning by using a CNN pretrained for audio tagging, treating the 1D airflow signals similarly to sound waves. These encoders act as feature extractors, mapping complex data into a shared embedding space. These embeddings are then passed through projection heads and concatenated into a single vector for the final classification head.

The study utilized a prospective dataset of 220 mechanically ventilated ICU patients from the UC Davis Medical Center. Ground truth diagnoses were adjudicated by three independent critical care physicians using the Berlin consensus criteria.

Results

The trimodal model achieved a superior AUROC of 0.86 and an overall accuracy of 78%. Ablation studies demonstrated that the trimodal approach significantly outperformed unimodal models, such as VWD alone (0.78 AUROC) and X-ray alone (0.76 AUROC).

Performance comparison across different data modalities and combinations for the entire cohort.

Interestingly, the combination of X-ray and Tabular data (0.85 AUROC) performed nearly as well as the full trimodal model, suggesting that EHR-derived tabular data carries substantial predictive weight. Subgroup analysis further revealed that the model performed exceptionally well on "clear-cut" cases (Group 1, 0.96 AUROC) but faced greater difficulty with complex cases involving comorbidities like heart failure (Group 2, 0.78 AUROC). This performance gap highlights that clinical ambiguity remains a major hurdle for automated diagnostic tools.

Discussion and Conclusion

This work demonstrates that transfer learning and multimodal fusion can enable high-performing deep learning models in the ICU despite small datasets. By incorporating continuous physiological signals like ventilator waveforms, the model captures dynamic nuances that static lab values might miss.

However, several limitations remain, including single-center bias and a relatively small sample size. The model also showed some difficulty with calibration at extreme probability ranges. Future work should focus on multi-center validation and the incorporation of additional modalities, such as clinical notes, to further close the gap in diagnosing complex, ambiguous cases. In conclusion, this trimodal framework serves as a robust blueprint for building performant models for conditions with small, heterogeneous datasets.

Summary updated

Abstract

Abstract Objective Poor outcomes in acute respiratory distress syndrome (ARDS) can be alleviated with tools that support early diagnosis. Current machine learning methods for detecting ARDS do not take full advantage of the multimodality of ARDS pathophysiology. We developed a multimodal deep learning model that uses imaging data, continuously collected ventilation data, and tabular data derived from a patient’s electronic health record (EHR) to make ARDS predictions. Materials and Methods A chest radiograph (x-ray), at least two hours of ventilator waveform (VWD) data within the first 24 hours of intubation, and EHR-derived tabular data were used from 220 patients admitted to the ICU to train a deep learning model. The model uses pretrained encoders for the x-rays and ventilation data and trains a feature extractor on tabular data. Encoded features for a patient are combined to make a single ARDS prediction. Ablation studies for each modality assessed their effect on the model’s predictive capability. Results The trimodal model achieved an area under the receiver operator curve (AUROC) of 0.86 with a 95% confidence interval of 0.01. This was a statistically significant improvement (p&lt;0.05) over single modality models and bimodal models trained on VWD+tabular and VWD+x-ray data. Discussion and Conclusion Our results demonstrate the potential utility of using deep learning to address complex conditions with heterogeneous data. More work is needed to determine the additive effect of modalities on ARDS detection. Our framework can serve as a blueprint for building performant multimodal deep learning models for conditions with small, heterogeneous datasets.
BibTeX
@misc{broecker2025multimodal,
  author = {Stefan Broecker and Jason Y. Adams and Girish Kumar and Rachael A. Callcut and Yuan Ni and Thomas Strohmer},
  title = {{Multimodal Deep Learning for ARDS Detection}},
  year = {2025},
  doi = {10.1101/2025.08.08.25333333},
  howpublished = {medRxiv},
}
Figure