Disorder-Free Data are All You Need: Inverse Supervised Learning for Broad-Spectrum Head Disorder Detection

Yuwei He, Yuchen Guo, Jinhao Lyu, Liangdi Ma, Haotian Tan, Wei Zhang, Guiguang Ding, Hengrui Liang, Jianxing He, Xin Lou, Qionghai Dai, Feng XumedRxiv 2023 · DOI

Abstract

BACKGROUND The development of artificial intelligence (AI)-based medical systems heavily relies on the collection and annotation of sufficient data containing disorders. However, the preparation of data with complete disorder types and adequate annotations presents a significant challenge, limiting the diagnostic capabilities of existing AI-based medical systems. This study introduces a novel AI-based system that accurately detects a broad spectrum of disorders without requiring any disorder-containing data. METHODS We obtained a training dataset of 21,429 disorder-free head computed tomography (CT) scans and proposed a learning algorithm called Inverse Supervised Learning (ISL). This algorithm learns and understands disorder-free samples instead of disorder-contained ones, enabling the identification of all types of disorders. We also developed a diagnosis and visualization software for clinical usage based on the system’s ability to provide visually understandable clues. RESULTS The system achieved Area Under the Curve (AUC) values of 0.883, 0.868, and 0.866 on retrospective (127 disorder types, 9,967 scans), prospective (117 disorder types, 3,054 scans), and cross-center (46 disorder types, 554 scans) datasets, respectively. These results demonstrate that the system can detect far more disorder types than previous AI-based systems. Furthermore, the ISL-based systems achieved AUC values of 0.893 and 0.895 on pulmonary CT and retinal optical coherence tomography (OCT), respectively, demonstrating that ISL can generalize well to non-head and non-CT images. CONCLUSIONS Our novel AI-based system, utilizing ISL, can accurately and broadly detect disorders without requiring disorder-containing data. This system not only outperforms previous AI-based systems in terms of disorder detection but also provides visually understandable clues, enhancing its clinical utility. The successful application of ISL to non-head and non-CT images further demonstrates its potential for broad-spectrum medical applications.

Introduction

The "data bottleneck" is a primary obstacle in medical AI. Standard supervised learning requires thousands of images of every specific disease, each meticulously labeled by expert radiologists. While effective for common conditions, this approach fails for the "long tail" of rare disorders. In clinical practice, a single brain CT scan can reveal over a hundred different types of abnormalities. Building separate models for each—or even one massive multi-class model—is practically impossible due to the scarcity of labeled data for rare conditions.

This paper proposes a "generalist" approach: instead of learning what every disease looks like, the AI learns exactly what "healthy" looks like and flags anything that doesn't fit. This novel system, utilizing Inverse Supervised Learning (ISL), accurately detects a broad spectrum of disorders without requiring any disorder-containing data for training.

Methods: Inverse Supervised Learning (ISL)

The core innovation is the ISL framework, which flips the traditional training script. The system consists of three main components:

  1. De-disorder Network (DeDN): Based on the computer vision task of image inpainting, this network is trained exclusively on healthy scans. During training, parts of a healthy image are masked, and the DeDN learns to fill them back in. Because it only knows "healthy," it will always attempt to reconstruct healthy tissue, even if the input image contains a lesion.
  2. Difference Mapping: When a scan with a disorder is fed into the DeDN, the network "fixes" the abnormal area, replacing it with what healthy tissue should look like. By subtracting the original image from this reconstructed "de-disordered" image, the system generates a heatmap of abnormalities.
  3. Disorder Recognition Network (DRN): To reduce noise and improve accuracy, a DRN analyzes the difference map. This network is trained using a "compactness loss" to ensure that healthy variations are tightly clustered in a latent space, while abnormalities are pushed far away.

Results and Generalization

The system demonstrated performance comparable to human experts across a massive variety of conditions. It achieved an average AUC of 0.883 on the retrospective head CT dataset, which covered 127 different disorder types. As shown in the performance table below, the system maintained high sensitivity and specificity across a wide array of pathologies, from common hemorrhages to rare fractures.

Performance metrics for various head disorders in the retrospective dataset, demonstrating high AUC across a broad spectrum of conditions.

Key performance highlights include:

  • Robust Generalization: The model maintained an AUC of 0.866 when tested on data from a completely different hospital (cross-center), proving it wasn't just "memorizing" the quirks of one imaging center.
  • Beyond the Brain: The ISL framework generalized seamlessly to other modalities, achieving AUCs of 0.893 for lung disorders and 0.895 for retinal diseases (OCT).
  • Explainability: Because the system works by comparing images, it naturally produces heatmaps that provide doctors with visual clues regarding the location of abnormalities.

Conclusion

Our novel AI-based system, utilizing ISL, can accurately and broadly detect disorders without requiring disorder-containing data. This system acts as a high-performance triage tool, flagging that something is wrong and where it is, even for rare conditions not explicitly seen during training. By bypassing the need for expensive, manual pixel-level annotations, ISL offers a scalable path toward generalist medical AI that can assist radiologists in high-pressure clinical environments.

Summary updated

Abstract

Abstract BACKGROUND The development of artificial intelligence (AI)-based medical systems heavily relies on the collection and annotation of sufficient data containing disorders. However, the preparation of data with complete disorder types and adequate annotations presents a significant challenge, limiting the diagnostic capabilities of existing AI-based medical systems. This study introduces a novel AI-based system that accurately detects a broad spectrum of disorders without requiring any disorder-containing data. METHODS We obtained a training dataset of 21,429 disorder-free head computed tomography (CT) scans and proposed a learning algorithm called Inverse Supervised Learning (ISL). This algorithm learns and understands disorder-free samples instead of disorder-contained ones, enabling the identification of all types of disorders. We also developed a diagnosis and visualization software for clinical usage based on the system’s ability to provide visually understandable clues. RESULTS The system achieved Area Under the Curve (AUC) values of 0.883, 0.868, and 0.866 on retrospective (127 disorder types, 9,967 scans), prospective (117 disorder types, 3,054 scans), and cross-center (46 disorder types, 554 scans) datasets, respectively. These results demonstrate that the system can detect far more disorder types than previous AI-based systems. Furthermore, the ISL-based systems achieved AUC values of 0.893 and 0.895 on pulmonary CT and retinal optical coherence tomography (OCT), respectively, demonstrating that ISL can generalize well to non-head and non-CT images. CONCLUSIONS Our novel AI-based system, utilizing ISL, can accurately and broadly detect disorders without requiring disorder-containing data. This system not only outperforms previous AI-based systems in terms of disorder detection but also provides visually understandable clues, enhancing its clinical utility. The successful application of ISL to non-head and non-CT images further demonstrates its potential for broad-spectrum medical applications. (Funded by National Key R&D Program of China, National Natural Science Foundation of China)
BibTeX
@misc{he2023disorder,
  author = {Yuwei He and Yuchen Guo and Jinhao Lyu and Liangdi Ma and Haotian Tan and Wei Zhang and Guiguang Ding and Hengrui Liang and Jianxing He and Xin Lou and Qionghai Dai and Feng Xu},
  title = {{Disorder-Free Data are All You Need: Inverse Supervised Learning for Broad-Spectrum Head Disorder Detection}},
  year = {2023},
  doi = {10.1101/2023.10.10.23296794},
  howpublished = {medRxiv},
}
Figure