Joint Learning of Generative Translator and Classifier for Visually Similar Classes

ByungIn Yoo, Tristan Sylvain, Yoshua Bengio, Junmo Kim

Abstract

In this paper, we propose a Generative Translation Classification Network (GTCN) for improving visual classification accuracy in settings where classes are visually similar and data is scarce. For this purpose, we propose joint learning from a scratch to train a classifier and a generative stochastic translation network end-to-end. The translation network is used to perform on-line data augmentation across classes, whereas previous works have mostly involved domain adaptation. To help the model further benefit from this data-augmentation, we introduce an adaptive fade-in loss and a quadruplet loss. We perform experiments on multiple datasets to demonstrate the proposed method's performance in varied settings. Of particular interest, training on 40% of the dataset is enough for our model to surpass the performance of baselines trained on the full dataset. When our architecture is trained on the full dataset, we achieve comparable performance with state-of-the-art methods despite using a light-weight architecture.

Introduction

Distinguishing between visually similar classes remains a significant challenge in computer vision, particularly when training data is limited. Tasks such as face liveness detection—differentiating a real human face from a high-quality print or screen—require the model to identify subtle decision boundaries. Standard neural networks often struggle with these "look-alike" classes unless provided with massive datasets, which are often expensive or impossible to acquire.

To address this, researchers have developed the Generative Translation Classification Network (GTCN). Unlike traditional approaches that train classifiers in isolation or rely on simple data augmentation (e.g., flipping or rotating), GTCN jointly learns a classifier alongside a generative stochastic translation network. This allows the model to generate synthetic "hard examples" on-the-fly, effectively teaching the classifier to navigate the most difficult regions of the feature space.

Methodology: The GTCN Framework

Inter-class Generative Translation

The core innovation of GTCN is the use of a translator to perform inter-class data augmentation. For instance, given an image of a cat (Class A), the translator learns to generate a version of that specific cat that possesses dog-like (Class B) characteristics. This process creates "augmented mini-batches" containing both real and translated hybrid images. As training progresses, these translated images become increasingly realistic, forcing the classifier to refine its decision boundary.

Conceptual overview of generative translation during training iterations

Architecture and Loss Functions

The GTCN architecture is designed to be lightweight, utilizing only six convolutional layers to ensure real-time performance on mobile devices. The joint training process is guided by several key components:

  1. Adaptive Fade-in (AF) Loss: Early in training, generative models often produce low-quality noise. The AF mechanism uses trainable parameters ($\alpha$ and $\beta$) to automatically down-weight the influence of these generated images initially, gradually increasing their weight as the translator improves.
  2. Quadruplet Loss (QL): This novel loss function ensures robust feature learning by forcing the network to keep real images and their translated counterparts close in the feature space while pushing images of different classes further apart.
Detailed GTCN architecture showing generators, discriminators, and the classifier with associated loss functions

Experimental Results

The GTCN was evaluated across three distinct datasets: Face Liveness Detection, Dogs vs. Cats, and an Artist Dataset (featuring paintings by Cezanne, Monet, Van Gogh, and Ukiyoe prints).

The "40% Rule"

One of the most significant findings is the model's efficiency. In the Face Liveness task (LV40 subset), GTCN achieved 92.26% accuracy using only 40% of the training data, significantly outperforming a standard CNN trained on the full 100% dataset, which reached only 74.71%.

Performance Benchmarks

  • Dogs vs. Cats: GTCN achieved an Equal Error Rate (EER) of 5.66%, surpassing VAEs, LSGANs, and other semi-supervised baselines.
  • Efficiency: The lightweight architecture allows for a processing time of just 30ms on a Galaxy S10, making it suitable for real-time mobile applications.
  • Feature Separation: t-SNE visualizations confirmed that GTCN produces tighter, more distinct clusters for similar classes compared to traditional training methods.

Conclusion and Future Work

GTCN demonstrates that joint learning with a generative translator can significantly boost classification performance in data-scarce environments. By focusing on the quality of training samples rather than model size, the framework achieves state-of-the-art results with a lightweight architecture.

While effective for binary and small-scale multi-class problems, scaling the translator to datasets with hundreds or thousands of classes (like ImageNet) remains a challenge. Future research will focus on "one-to-many" translation architectures to maintain this performance advantage at a larger scale.

Summary updated

Abstract

In this paper, we propose a Generative Translation Classification Network (GTCN) for improving visual classification accuracy in settings where classes are visually similar and data is scarce. For this purpose, we propose joint learning from a scratch to train a classifier and a generative stochastic translation network end-to-end. The translation network is used to perform on-line data augmentation across classes, whereas previous works have mostly involved domain adaptation. To help the model further benefit from this data-augmentation, we introduce an adaptive fade-in loss and a quadruplet loss. We perform experiments on multiple datasets to demonstrate the proposed method's performance in varied settings. Of particular interest, training on 40% of the dataset is enough for our model to surpass the performance of baselines trained on the full dataset. When our architecture is trained on the full dataset, we achieve comparable performance with state-of-the-art methods despite using a light-weight architecture.
BibTeX
@inproceedings{yoo2020joint,
  author = {ByungIn Yoo and Tristan Sylvain and Yoshua Bengio and Junmo Kim},
  title = {{Joint Learning of Generative Translator and Classifier for Visually Similar Classes}},
  booktitle = {IEEE Access},
  year = {2020},
  url = {https://openreview.net/forum?id=9s1d2E9VxbI},
}
Figure