DT-MIL: Deformable Transformer for Multi-instance Learning on Histopathological Image
Lists
Only you see this
Abstract
Learning informative representations is crucial for classification and prediction tasks on histopathological images. Due to the huge image size, whole-slide histopathological image analysis is normally addressed with multi-instance learning (MIL) scheme. However, the weakly supervised nature of MIL leads to the challenge of learning an effective whole-slide-level representation. To tackle this issue, we present a novel embedded-space MIL model based on deformable transformer (DT) architecture and convolutional layers, which is termed DT-MIL. The DT architecture enables our MIL model to update each instance feature by globally aggregating instance features in a bag simultaneously and encoding the position context information of instances during bag representation learning. Compared with other state-of-the-art MIL models, our model has the following advantages: (1) generating the bag representation in a fully trainable way, (2) representing the bag with a high-level and nonlinear combination of all instances instead of fixed pooling-based methods (e.g. max pooling and average pooling) or simply attention-based linear aggregation, and (3) encoding the position relationship and context information during bag embedding phase. Besides our proposed DT-MIL, we also develop other possible transformer-based MILs for comparison. Extensive experiments show that our DT-MIL outperforms the state-of-the-art methods and other transformer-based MIL architectures in histopathological image classification and prediction tasks. An open-source implementation of our approach can be found at https://github.com/yfzon/DT-MIL .
BibTeX
@inproceedings{li2021dt,
author = {Hang Li and Fan Yang and Yu Zhao and Xiaohan Xing and Jun Zhang and Mingxuan Gao and Junzhou Huang and Liansheng Wang and Jianhua Yao},
title = {{DT-MIL: Deformable Transformer for Multi-instance Learning on Histopathological Image}},
booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
year = {2021},
doi = {10.1007/978-3-030-87237-3\_20},
}Loading related content...