A Plug-in Method for Representation Factorization in Connectionist Models

Jee Seok Yoon, Myung-Cheol Roh, Heung-Il Suk

Abstract

In this article, we focus on decomposing latent representations in generative adversarial networks or learned feature representations in deep autoencoders into semantically controllable factors in a semisupervised manner, without modifying the original trained models. Particularly, we propose factors’ decomposer-entangler network (FDEN) that learns to decompose a latent representation into mutually independent factors. Given a latent representation, the proposed framework draws a set of interpretable factors, each aligned to independent factors of variations by minimizing their total correlation in an information-theoretic means. As a plug-in method, we have applied our proposed FDEN to the existing networks of adversarially learned inference and pioneer network and performed computer vision tasks of image-to-image translation in semantic ways, e.g., changing styles, while keeping the identity of a subject, and object classification in a few-shot learning scheme. We have also validated the effectiveness of the proposed method with various ablation studies in the qualitative, quantitative, and statistical examination.
BibTeX
@article{yoon2022plug,
  author = {Jee Seok Yoon and Myung-Cheol Roh and Heung-Il Suk},
  title = {{A Plug-in Method for Representation Factorization in Connectionist Models}},
  journal = {IEEE TNNLS},
  year = {2022},
  doi = {10.1109/tnnls.2021.3054480},
}

Loading related content...

Figure