Deep Metric Network Via Heterogeneous Semantics for Image Sentiment Analysis

Yun Liang, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama

Abstract

This paper presents a novel method for image sentiment analysis called a deep metric network via heterogeneous semantics (DMNHS). The contribution of the proposed method is introduction of the image captioning into image sentiment analysis to reflect a global impression that cannot be represented by classical visual features extracted from images. In order to consider a sentiment correlation between visual and captioning features, the proposed method newly designs a network to integrate these heterogeneous semantics features (HS features). Furthermore, with consideration of relations among sentiments based on the HS features, the proposed method constructs a sentiment latent space by introducing the center loss concerning relationships between different sentiments and enables the classification of image sentiments. From experimental results, the performance improvement via DMN-HS is confirmed.
BibTeX
@inproceedings{liang2021deep,
  author = {Yun Liang and Keisuke Maeda and Takahiro Ogawa and Miki Haseyama},
  title = {{Deep Metric Network Via Heterogeneous Semantics for Image Sentiment Analysis}},
  booktitle = {ICIP},
  year = {2021},
  doi = {10.1109/icip42928.2021.9506701},
}

Loading related content...

Figure