Deep Representation Learning Characterized by Inter-Class Separation for Image Clustering

Dipanjan Das, Ratul Ghosh, Brojeshwar Bhowmick

Clustering images—grouping them by content without any labels—is a "holy grail" of computer vision. While humans can instantly see the difference between a cat and a car, machines often get bogged down in pixel-level noise. A new paper from researchers at TCS Research, titled "Deep Representation Learning Characterized by Inter-class Separation for Image Clustering," proposes a way to make these machine-learned groupings much more distinct.

TL;DR

The researchers developed a deep learning framework that simultaneously learns how to represent images and how to cluster them. By using a novel "triplet" constraint and a smart negative-sampling algorithm, their model ensures that similar images stay close together while dissimilar ones are pushed far apart in the data's "latent space," leading to state-of-the-art clustering accuracy.

What Problem Does This Paper Address?

Traditional clustering algorithms like K-means often fail on natural images because they rely on raw pixel distances, which don't capture semantic meaning. To solve this, researchers usually turn to Deep Learning to compress images into "features."

However, two major hurdles remain:

  1. The Separation Gap: Often, feature learning and clustering are done separately. This means the features aren't "aware" they need to be clustered, leading to messy boundaries where different classes overlap.
  2. The Supervision Trap: Many high-performing models rely on pre-trained networks (like those trained on ImageNet with labels). This limits their ability to generalize to entirely new types of data where labels don't exist.

The authors aim to create a system that learns discriminative features from scratch in a purely unsupervised way.

Methods

The core of the approach is a Deep Convolutional Autoencoder (DCA). An autoencoder learns to compress an image into a small "bottleneck" vector (the representation) and then reconstruct the original image from that vector.

The team introduced three key innovations to the standard training process:

  1. The Inter-class Separation Constraint ($L_t$): Inspired by triplet loss in supervised learning, the model takes three inputs: an Anchor image, a Positive image (an augmented version of the anchor, like a rotated or flipped version), and a Negative image. The goal is to minimize the distance between the anchor and positive while maximizing the distance to the negative.
  2. Smart Negative Selection: In unsupervised learning, you don't know which images are "negatives." If you pick a random image, it might actually be from the same class as your anchor. The authors developed an algorithm that selects a negative image only if its representation falls outside the distribution of the anchor's own augmented variations.
  3. Entropy Minimization ($L_b$): To prevent "cluster collapse"—where the model gets lazy and assigns almost every image to the same few clusters—they added a constraint that encourages the model to make unambiguous, confident assignments across all available clusters.

Datasets

The model was evaluated on several standard benchmarks:

  • MNIST & USPS: Classic handwritten digit datasets.
  • CIFAR-10 & CIFAR-100: Small color images of objects and animals.
  • FRGC: A face recognition dataset.
  • ImageNet: The authors selected 10 random classes to test the model's performance on high-resolution, complex natural images.

Key Insights and Ideas

  • Joint Optimization is Key: By training the autoencoder and the clustering layer at the same time, the features naturally evolve to be "cluster-friendly."
  • Augmentation as a Proxy for Labels: Since the model knows that a rotated "7" is still a "7," it uses data augmentation to define what "similarity" looks like without needing a human to provide a label.
  • Pushing is as Important as Pulling: Most unsupervised methods focus on bringing similar things together. This paper highlights that explicitly pushing dissimilar things away (inter-class separation) is what truly cleans up the cluster boundaries.

Key Results

The proposed method achieved impressive quantitative results, often outperforming previous state-of-the-art (SOTA) methods:

  • MNIST: Achieved 98.93% accuracy, beating other deep clustering methods like DEC (84%) and JULE (96.4%).
  • USPS: Reached 97.63% accuracy.
  • FRGC (Faces): Achieved 47.28%, a significant jump over K-means (24.3%).
  • Generalization: Unlike some competitors (like IMSAT), this model does not rely on pre-trained weights from other datasets to perform well on CIFAR, proving it is more robust for "cold-start" scenarios.
  • ImageNet: The authors are among the first to report results on a subset of ImageNet for this type of unsupervised clustering, achieving 35.69% accuracy.

Limitations and Future Work

While the model performs exceptionally well on structured data like digits and faces, the authors note a performance gap on "feature-rich" datasets like CIFAR-100 compared to methods that use supervised pre-training.

The main limitation identified is that the "auxiliary distribution" (the model's internal guess at where clusters should be) isn't always perfectly aligned with the true data distribution. The authors suggest that a more sophisticated, separate learning strategy for this auxiliary distribution could be a fruitful path for future research, especially for complex datasets like the full ImageNet.

Summary updated

Abstract

Despite significant advances in clustering methods in recent years, the outcome of clustering of a natural image dataset is still unsatisfactory due to two important drawbacks. Firstly, clustering of images needs a good feature representation of an image and secondly, we need a robust method which can discriminate these features for making them belonging to different clusters such that intra-class variance is less and inter-class variance is high. Often these two aspects are dealt with independently and thus the features are not sufficient enough to partition the data meaningfully. In this paper, we propose a method where we discover these features required for the separation of the images using deep autoencoder. Our method learns the image representation features automatically for the purpose of clustering and also select a coherent image and an incoherent image simultaneously for a given image so that the feature representation learning can learn better discriminative features for grouping the similar images in a cluster and at the same time separating the dissimilar images across clusters. Experiment results show that our method produces significantly better result than the state-of-the-art methods and we also show that our method is more generalized across different dataset without using any pre-trained model like other existing methods.
BibTeX
@inproceedings{das2019deep,
  author = {Dipanjan Das and Ratul Ghosh and Brojeshwar Bhowmick},
  title = {{Deep Representation Learning Characterized by Inter-Class Separation for Image Clustering}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2019},
  url = {https://openreview.net/forum?id=0i9MXDxOcKl},
}

Loading related content...

Figure