Subject2Vec: Generative-Discriminative Approach from a Set of Image Patches to a Vector
Lists
Only you see this
Abstract
We propose an attention-based method that aggregates local image features to a subject-level representation for predicting disease severity. In contrast to classical deep learning that requires a fixed dimensional input, our method operates on a set of image patches; hence it can accommodate variable length input image without image resizing. The model learns a clinically interpretable subject-level representation that is reflective of the disease severity. Our model consists of three mutually dependent modules which regulate each other: (1) a discriminative network that learns a fixed-length representation from local features and maps them to disease severity; (2) an attention mechanism that provides interpretability by focusing on the areas of the anatomy that contribute the most to the prediction task; and (3) a generative network that encourages the diversity of the local latent features. The generative term ensures that the attention weights are non-degenerate while maintaining the relevance of the local regions to the disease severity. We train our model end-to-end in the context of a large-scale lung CT study of Chronic Obstructive Pulmonary Disease (COPD). Our model gives state-of-the art performance in predicting clinical measures of severity for COPD. The distribution of the attention provides the regional relevance of lung tissue to the clinical measurements.
BibTeX
@inproceedings{singla2018subject,
author = {Sumedha Singla and Mingming Gong and Siamak Ravanbakhsh and Frank C. Sciurba and Barnabás Póczos and Kayhan N. Batmanghelich},
title = {{Subject2Vec: Generative-Discriminative Approach from a Set of Image Patches to a Vector}},
booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
year = {2018},
doi = {10.1007/978-3-030-00928-1\_57},
}