Multi-View Abnormality Detection in Clinical Knee MRI Studies Using Transformers

Firas Khader, Johannes Stegmaier, Sven Nebelung, Daniel Truhn

Abstract

We propose a novel neural network architecture based on transformers that is capable of combining information from multiple three-dimensional Magnetic Resonance Imaging (MRI) scans taken in different views, i.e. sagittal, coronal and axial. We demonstrate its performance on a public dataset and show that it is on par with other state-of-the-art methods in terms of area under curve (0.93 ± 0.01), sensitivity (0.86 ± 0.02), specificity (0.89 ± 0.05) and accuracy (0.86 ± 0.01) in detecting abnormalities. We further show that robustness to missing data can be increased when applying multi-view dropout and outline the potential of such an architecture in the field of medical imaging.
BibTeX
@inproceedings{khader2023multi,
  author = {Firas Khader and Johannes Stegmaier and Sven Nebelung and Daniel Truhn},
  title = {{Multi-View Abnormality Detection in Clinical Knee MRI Studies Using Transformers}},
  booktitle = {ISBI},
  year = {2023},
  doi = {10.1109/isbi53787.2023.10230643},
}

Loading related content...

Figure