Abstract
In real-world clinical practice, overlooking unanticipated findings can result in serious consequences. However, supervised learning, which is the foundation for the current success of deep learning, only encourages models to identify abnormalities that are defined in datasets in advance. Therefore, abnormality detection must be implemented in medical images that are not limited to a specific disease category. In this study, we demonstrate an unsupervised learning framework for pixel-wise abnormality detection in brain magnetic resonance imaging captured from a patient population with metastatic brain tumor. Our concept is as follows: If an image reconstruction network can faithfully reproduce the global features of normal anatomy, then the abnormal lesions in unseen images can be identified based on the local difference from those reconstructed as normal by a discriminative network. Both networks are trained on a dataset comprising only normal images without labels. In addition, we devise a metric to evaluate the anatomical fidelity of the reconstructed images and confirm that the overall detection performance is improved when the image reconstruction network achieves a higher score. For evaluation, clinically significant abnormalities are comprehensively segmented. The results show that the area under the receiver operating characteristics curve values for metastatic brain tumors, extracranial metastatic tumors, postoperative cavities, and structural changes are 0.78, 0.61, 0.91, and 0.60, respectively.
Introduction
In clinical radiology, the ability to detect unanticipated findings is critical. While supervised deep learning has achieved significant success in identifying specific diseases, it remains limited by its training data; models can typically only identify pathologies they have been explicitly taught to recognize. Creating comprehensive labeled datasets for the vast array of human pathologies is impractical due to the high cost of expert annotation and the sheer variety of conditions.
This paper addresses the challenge of unsupervised abnormality detection. The goal is to develop a model that learns the distribution of "normal" brain anatomy so thoroughly that it can flag any deviation—such as tumors, surgical cavities, or structural shifts—as a potential abnormality, regardless of whether that specific condition was encountered during training.
Methods
The proposed framework utilizes a two-step pipeline to address both global structural changes and local textural anomalies.
Global Reconstruction with IntroVAE and Latent Search
Standard Variational Autoencoders (VAEs) often produce blurry reconstructions, which can obscure subtle anatomical details. To overcome this, the authors employ an Introspective VAE (IntroVAE). This architecture uses an adversarial-like min-max game to encourage the generation of high-resolution, realistic images.
To further enhance anatomical accuracy, the authors implement Latent Representation Search. This iterative process fine-tunes the latent vector $z$ to ensure the reconstructed "normal" version of a patient's brain is as faithful as possible to the input's global structure.
Local Discrimination and Feature Embedding
Rather than relying on simple pixel-wise subtraction, which is sensitive to noise and minor misalignments, the system uses a Discriminative Network. This network is trained using a triplet loss on normal brain patches to map anatomical features into a specific embedding space. During inference, an "abnormality score" is calculated based on the distance between the feature embeddings of the original image and its reconstructed normal counterpart.
Results and Discussion
The model was trained on 29,278 axial MRI slices from 200 healthy subjects and tested on 88 MRI volumes from 75 patients with various clinical findings. Evaluation was performed using the Area Under the Receiver Operating Characteristic curve (AUC-ROC) across four categories of abnormalities.
The model demonstrated strong performance in identifying major structural deviations:
- Postoperative Cavities: 0.91 AUC
- Metastatic Brain Tumors: 0.78 AUC
- Extracranial Tumors: 0.61 AUC
- Structural Changes (NOS): 0.60 AUC
The results indicate that while the model excels at detecting large structural voids and focal lesions, subtle structural shifts and extracranial findings remain challenging. A key insight from the study is that detection performance correlates with "anatomical fidelity"—as the quality of the normal reconstruction improves, so does the model's ability to distinguish abnormalities.
Conclusion
This research presents a robust unsupervised framework for brain abnormality detection. By focusing on high-fidelity reconstruction of normal anatomy and comparing features in a learned embedding space, the model can identify a variety of clinical findings without prior labels. Future work will focus on improving precision and exploring more advanced generative architectures to further enhance reconstruction quality and detection sensitivity.
Summary updated