Joint Learning of Generative Translator and Classifier for Visually Similar Classes

ByungIn Yoo, Tristan Sylvain, Yoshua Bengio, Junmo Kim

TL;DR

Researchers have developed the Generative Translation Classification Network (GTCN), a framework that jointly trains a image classifier alongside a generative "translator" to solve the problem of distinguishing visually similar classes with limited data. By creating synthetic "hard examples" on-the-fly, the model achieves higher accuracy with just 40% of a dataset than standard models do with 100%.

What Problem Does This Paper Address?

In computer vision, two of the toughest hurdles are visual similarity and data scarcity.

Consider "face liveness detection"—telling the difference between a real human face and a high-quality printed photo or a video screen. To a standard neural network, these look nearly identical. Distinguishing them requires a massive amount of data to find the subtle "decision boundary." However, collecting thousands of high-quality "spoof" images is expensive and time-consuming.

Existing solutions often rely on massive, computationally heavy models that don't run well on smartphones, or simple data augmentation (like flipping images) that doesn't help the network learn the deep structural differences between two very similar classes.

Methods

The authors propose the Generative Translation Classification Network (GTCN). Instead of training a classifier in isolation, they train it alongside a Generative Stochastic Translation Network.

1. Inter-class Translation

The core idea is "translation." If you have an image of a cat (Class A) and a dog (Class B), the translator learns to generate a version of that specific cat that looks like a dog, and vice versa. This creates "augmented mini-batches" where half the images are real and half are these "translated" hybrids.

2. Adaptive Fade-in (AF) Loss

Early in training, generative models produce blurry, low-quality junk. If the classifier relies on this junk, it learns poorly. The authors introduced an Adaptive Fade-in mechanism that uses trainable parameters ($\alpha$ and $\beta$) to automatically down-weight the influence of generated images early on, gradually increasing their importance as the translator gets better.

3. Quadruplet Loss (QL)

To ensure the classifier learns the right features, the authors designed a novel quadruplet loss. This loss function explicitly forces the network to:

  • Keep real images and their translated counterparts (e.g., a real face and its "fake" translation) close in the feature space.
  • Push images of different classes (e.g., real faces vs. actual fake faces) far apart.

Datasets

The model was evaluated on three distinct challenges:

  • Face Liveness Detection: A dataset containing real faces and three types of "fakes" (video replay, printed photos, and masks).
  • Dogs vs. Cats: A classic binary classification task where the two classes share many visual features (fur, ears, eyes).
  • Artist Dataset: A multi-class dataset featuring paintings by Cezanne, Monet, Van Gogh, and Ukiyoe prints, where styles often overlap.

Key Insights and Ideas

  • Generative Models as Teachers: Instead of just using GANs to make pretty pictures, this paper uses them to create "hard" training samples that sit right on the edge of the decision boundary. This forces the classifier to work harder and learn more robust features.
  • Efficiency over Size: The authors used a very light-weight architecture (only 6 convolutional layers). By focusing on how the model learns (jointly with a translator) rather than how big the model is, they achieved state-of-the-art results that can run in real-time on a smartphone (30ms on a Galaxy S10).
  • The "40% Rule": Perhaps the most striking insight is that the "translated" data is so effective that the model trained on only 40% of the available data outperformed a standard CNN trained on the full 100%.

Key Results

  • Face Liveness: On the LV40 subset (only 8 subjects), GTCN achieved 92.26% accuracy, compared to just 74.71% for a standard CNN.
  • Low Error Rates: In the Dogs vs. Cats task, GTCN achieved an Equal Error Rate (EER) of 5.66%, outperforming VAEs, LSGANs, and semi-supervised baselines.
  • Superior Separation: t-SNE visualizations (a way to map high-dimensional data into 2D) showed that GTCN creates much tighter, more distinct clusters for similar classes compared to traditional training methods.

Limitations and Future Work

While the paper demonstrates excellent results for binary and small-scale multi-class problems (like the 4-artist set), the complexity of the "translator" might scale poorly as the number of classes increases significantly (e.g., ImageNet's 1,000 classes). Training a translator for every possible pair of classes would be computationally prohibitive.

Future work could explore "one-to-many" or "many-to-many" translation architectures to make GTCN more scalable for massive datasets while maintaining its impressive performance in data-scarce environments.

Summary updated

Abstract

In this paper, we propose a Generative Translation Classification Network (GTCN) for improving visual classification accuracy in settings where classes are visually similar and data is scarce. For this purpose, we propose joint learning from a scratch to train a classifier and a generative stochastic translation network end-to-end. The translation network is used to perform on-line data augmentation across classes, whereas previous works have mostly involved domain adaptation. To help the model further benefit from this data-augmentation, we introduce an adaptive fade-in loss and a quadruplet loss. We perform experiments on multiple datasets to demonstrate the proposed method's performance in varied settings. Of particular interest, training on 40% of the dataset is enough for our model to surpass the performance of baselines trained on the full dataset. When our architecture is trained on the full dataset, we achieve comparable performance with state-of-the-art methods despite using a light-weight architecture.
BibTeX
@inproceedings{yoo2020joint,
  author = {ByungIn Yoo and Tristan Sylvain and Yoshua Bengio and Junmo Kim},
  title = {{Joint Learning of Generative Translator and Classifier for Visually Similar Classes}},
  booktitle = {IEEE Access},
  year = {2020},
  url = {https://openreview.net/forum?id=9s1d2E9VxbI},
}

Loading related content...

Figure