Mutual Information Regularized Feature-Level Frankenstein for Discriminative Recognition

Xiaofeng Liu, Chao Yang, Jane You, C.-c. Jay Kuo, B. V. K. Vijaya Kumar

Abstract

Deep learning recognition approaches can potentially perform better if we can extract a discriminative representation that controllably separates nuisance factors. In this paper, we propose a novel approach to explicitly enforce the extracted discriminative representation <inline-formula><tex-math notation="LaTeX">$\boldsymbol{d}$</tex-math></inline-formula> , extracted latent variation <inline-formula><tex-math notation="LaTeX">$\boldsymbol{l}$</tex-math></inline-formula> (e,g., background, unlabeled nuisance attributes), and semantic variation label vector <inline-formula><tex-math notation="LaTeX">$\boldsymbol{s}$</tex-math></inline-formula> (e.g., labeled expressions/pose) to be independent and complementary to each other. We can cast this problem as an adversarial game in the latent space of an auto-encoder. Specifically, with the to-be-disentangled <inline-formula><tex-math notation="LaTeX">$\boldsymbol{s}$</tex-math></inline-formula> , we propose to equip an end-to-end conditional adversarial network with the ability to decompose an input sample into <inline-formula><tex-math notation="LaTeX">${\boldsymbol{d}}$</tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX">$\boldsymbol{l}$</tex-math></inline-formula> . However, we argue that maximizing the cross-entropy loss of semantic variation prediction from <inline-formula><tex-math notation="LaTeX">$\boldsymbol{d}$</tex-math></inline-formula> is not sufficient to remove the impact of <inline-formula><tex-math notation="LaTeX">$\boldsymbol{s}$</tex-math></inline-formula> from <inline-formula><tex-math notation="LaTeX">$\boldsymbol{d}$</tex-math></inline-formula> , and that the uniform-target and entropy regularization are necessary. A collaborative mutual information regularization framework is further proposed to avoid unstable adversarial training. It is able to minimize the differentiable mutual information between the variables to enforce independence. The proposed discriminative representation inherits the desired tolerance property guided by prior knowledge of the task. Our proposed framework achieves top performance on diverse recognition tasks, including digits classification, large-scale face recognition on LFW and IJB-A datasets, and face recognition tolerant to changes in lighting, makeup, disguise, etc.
BibTeX
@article{liu2022mutual,
  author = {Xiaofeng Liu and Chao Yang and Jane You and C.-C. Jay Kuo and B. V. K. Vijaya Kumar},
  title = {{Mutual Information Regularized Feature-Level Frankenstein for Discriminative Recognition}},
  journal = {IEEE Trans. Pattern Anal. Mach. Intell.},
  year = {2022},
  url = {https://openreview.net/forum?id=jZnMZBJ-Ka},
}

Loading related content...

Figure