Doubly Semi-Supervised Multimodal Adversarial Learning for Classification, Generation and Retrieval
Lists
Only you see this
Abstract
Learning over incomplete multi-modality data is a challenging problem with strong practical applications. Most existing multi-modal data imputation approaches have two limitations: (1) they are unable to accurately control the semantics of imputed modalities; and (2) without a shared low-dimensional latent space, they do not scale well with multiple modalities. To overcome the limitations, we propose a novel doubly semi-supervised multi-modal learning framework (DSML) with a modality-shared latent space and modality-specific generators, encoders and classifiers. We design novel softmax-based discriminators to train all modules adversarially. As a unified framework, DSML can be applied in multi-modal semi-supervised classification, missing modality imputation and fast cross-modality retrieval tasks simultaneously. Experiments on multiple datasets demonstrate its advantages.
BibTeX
@inproceedings{du2019doubly,
author = {Changde Du and Changying Du and Huiguang He},
title = {{Doubly Semi-Supervised Multimodal Adversarial Learning for Classification, Generation and Retrieval}},
booktitle = {ICME},
year = {2019},
doi = {10.1109/icme.2019.00011},
}Loading related content...