Attentive Recurrent Neural Network for Weak-supervised Multi-label Image Classification

Liang Li, Shuhui Wang, Shuqiang Jiang, Qingming Huang

Abstract

Multi-label image classification is a fundamental and challenging task in computer vision, and recently achieved significant progress by exploiting semantic relations among labels. However, the spatial positions of labels for multi-labels images are usually not provided in real scenarios, which brings insuperable barrier to conventional models. In this paper, we propose an end-to-end attentive recurrent neural network for multi-label image classification under only image-level supervision, which learns the discriminative feature representations and models the label relations simultaneously. First, inspired by attention mechanism, we propose a recurrent highlight network (RHN) which focuses on the most related regions in the image to learn the discriminative feature representations for different objects in an iterative manner. Second, we develop a gated recurrent relation extractor (GRRE) to model the label relations using multiplicative gates in a recurrent fashion, which learns to decide how multiple labels of the image influence the relation extraction. Extensive experiments on three benchmark datasets show that our model outperforms the state-of-the-arts, and performs better on small-object categories and under the scenario with large number of labels.
BibTeX
@inproceedings{li2018attentive,
  author = {Liang Li and Shuhui Wang and Shuqiang Jiang and Qingming Huang},
  title = {{Attentive Recurrent Neural Network for Weak-supervised Multi-label Image Classification}},
  booktitle = {ACM Multimedia},
  year = {2018},
  doi = {10.1145/3240508.3240649},
}

Loading related content...

Figure