Enhancing Interstitial Lung Disease Diagnoses Through Multimodal <scp>AI</scp> Integration of Histopathological and <scp>CT</scp> Image Data

Kris Lami, Mutsumi Ozasa, X. Che, Wataru Uegami, Yoshihiro Kato, Yoshiaki Zaizen, Naoko Tsuyama, Ichiro Mori, Shin Ichihara, Han‐Seung Yoon, Ryoko Egashira, Kensuke Kataoka, Takeshi Johkoh, Yasuhiro Kondo, Richard Attanoos, Alberto Cavazza, Alberto M. Marchevsky, Frank Schneider, Jaroslaw Augustyniak, Amna Almutrafi, Alexandre Todorovic Fabro, Luka Brčić, Anja C. Roden, Maxwell L. Smith, André L. Moreira, Junya FukuokaRespirology 2025 · DOI

TL;DR

This paper introduces a multimodal AI system that combines CT scans and histopathology images to improve the diagnosis of Usual Interstitial Pneumonia (UIP), a common and serious form of Interstitial Lung Disease (ILD). The AI significantly boosts diagnostic accuracy and confidence, especially for general pathologists, by integrating two key diagnostic modalities.

What Problem Does This Paper Address?

Diagnosing Interstitial Lung Diseases (ILDs) is notoriously difficult. It requires integrating clinical, radiological (CT scans), and histopathological (tissue biopsy) data, often necessitating a multidisciplinary discussion. A key challenge is accurately identifying Usual Interstitial Pneumonia (UIP), a pattern associated with poor prognosis. Even expert pathologists and radiologists struggle with consistent UIP diagnosis due to overlapping features and the need for specialized expertise. This inconsistency makes it hard for general pathologists to confidently diagnose ILDs, often leading to cases being concentrated in specialized centers. The paper aims to bridge this gap by developing an AI that can reliably integrate CT and histopathology data to aid in UIP diagnosis, making it more accessible and accurate.

Methods

The study developed a multimodal AI by combining two components:

  • CT-AI: This component analyzes CT images. It involves:
  • Feature Extraction: Segmenting lung fields into millions of 32x32 pixel patches.
  • Clustering: Using k-means clustering to group similar patches and then classifying these clusters into 8 distinct radiological patterns.
  • Classification: Two ILD specialists classified these patterns into UIP and non-UIP categories.
  • AI Model: A random forest model was trained on 28 extracted features to predict UIP presence, outputting a probability score.
  • Pathology-AI (MIXTURE): This component, developed in a prior study, analyzes histopathological images (whole slide images) to predict UIP.
  • Multimodal AI: The CT-AI and Pathology-AI outputs were combined using a Random Forest model to generate a final UIP probability score.

The performance of the multimodal AI was evaluated by comparing its predictions to those of expert pathologists and general pathologists, both before and after AI assistance.

Datasets

The study utilized a dataset of ILD cases collected between 2009 and 2021 from Tosei General Hospital, Japan.

  • Initial Cohort: 770 cases diagnosed with ILD after multidisciplinary discussion.
  • Exclusions: Cases with no volumetric CT, pre-biopsy CT, or clear pathological diagnosis were excluded.
  • Final Dataset: 324 patients with ILD and corresponding CT and pathological images were used.
  • 74 cases for CT image feature extraction.
  • 93 cases for CT image annotation.
  • 61 cases for multimodal AI model training.
  • 114 cases for multimodal AI model testing.

The study also involved 4 general pathologists and 11 ILD specialists for validation.

Key Insights and Ideas

  • Synergy of Modalities: The core idea is that combining CT and histopathology data provides a more comprehensive picture than either modality alone, leading to improved diagnostic accuracy for complex diseases like ILD.
  • Democratizing Expertise: The AI acts as a powerful assistant, significantly improving the diagnostic accuracy and confidence of general pathologists, who may lack specialized ILD expertise. This could help standardize diagnoses across different institutions.
  • Interpretable AI: The CT-AI component provides a "heatmap" visualization, offering some interpretability by showing where in the lung the AI is detecting UIP-related features. The Pathology-AI (MIXTURE) is also designed for explainability.
  • Beyond UIP: While the study focuses on UIP, the multimodal approach holds promise for diagnosing other complex ILDs.

Key Results

  • Multimodal AI Performance: The multimodal AI achieved a high overall accuracy of 82.37% ±2.81% for detecting UIP in the test set.
  • AUC: The model demonstrated a strong Area Under the Curve (AUC) of 0.9238 ±0.0152 for detecting UIP when compared to expert pathologist diagnoses.
  • Impact on General Pathologists:
    • Diagnostic concordance among general pathologists improved significantly from a κ score of 0.273 to 0.737 after AI assistance.
  • Diagnostic confidence increased, with three out of four general pathologists reporting high confidence in about 50% of cases after using the AI, compared to only 10% previously.
  • Agreement with expert consensus also improved.
  • CT-AI Performance: The CT-AI component alone achieved an AUC of 0.8492 ±0.0423 for UIP classification.

Limitations and Future Work

  • Single Institution Data: The multimodal AI model was trained and evaluated on data from a single medical institution, which may limit its generalizability. Data augmentation was used to mitigate this, but external validation is needed.
  • Limited Scanner/Scanner Types: While multiple CT scanner models were used, histopathological analysis was restricted to a single digital slide scanner.
  • Potential for Bias: Annotations by pathologists and radiologists, even with consensus, may introduce some degree of bias.
  • Future Directions:
    • External Validation: Testing the model on data from multiple institutions is crucial.
  • Broader Biopsy Techniques: Extending the methodology to other biopsy techniques like cryobiopsy.
  • Diverse Data Sources: Incorporating data from more diverse medical sources and facilities.
  • Clinical Data Integration: While the paper focuses on CT and histopathology, integrating clinical data could further enhance diagnostic capabilities.
  • Explainability: While efforts are made, further enhancing the explainability of AI decisions is always beneficial.

Summary updated

Abstract

Combining CT and histopathological images, the multimodal AI algorithm enhances pathologists' diagnostic accuracy, consistency, and confidence in identifying UIP, even in cases where specialised expertise is limited.
BibTeX
@inproceedings{lami2025enhancing,
  author = {Kris Lami and Mutsumi Ozasa and X. Che and Wataru Uegami and Yoshihiro Kato and Yoshiaki Zaizen and Naoko Tsuyama and Ichiro Mori and Shin Ichihara and Han‐Seung Yoon and Ryoko Egashira and Kensuke Kataoka and Takeshi Johkoh and Yasuhiro Kondo and Richard Attanoos and Alberto Cavazza and Alberto M. Marchevsky and Frank Schneider and Jaroslaw Augustyniak and Amna Almutrafi and Alexandre Todorovic Fabro and Luka Brčić and Anja C. Roden and Maxwell L. Smith and André L. Moreira and Junya Fukuoka},
  title = {{Enhancing Interstitial Lung Disease Diagnoses Through Multimodal <scp>AI</scp> Integration of Histopathological and <scp>CT</scp> Image Data}},
  booktitle = {Respirology},
  year = {2025},
  doi = {10.1111/resp.70036},
}
Figure