Deep Representation Learning Characterized by Inter-class Separation for Image Clustering

Dipanjan Das, Ratul Ghosh, Brojeshwar Bhowmick

Abstract

Despite significant advances in clustering methods in recent years, the outcome of clustering of a natural image dataset is still unsatisfactory due to two important drawbacks. Firstly, clustering of images needs a good feature representation of an image and secondly, we need a robust method which can discriminate these features for making them belonging to different clusters such that intra-class variance is less and inter-class variance is high. Often these two aspects are dealt with independently and thus the features are not sufficient enough to partition the data meaningfully. In this paper, we propose a method where we discover these features required for the separation of the images using deep autoencoder. Our method learns the image representation features automatically for the purpose of clustering and also select a coherent image and an incoherent image simultaneously for a given image so that the feature representation learning can learn better discriminative features for grouping the similar images in a cluster and at the same time separating the dissimilar images across clusters. Experiment results show that our method produces significantly better result than the state-of-the-art methods and we also show that our method is more generalized across different dataset without using any pre-trained model like other existing methods.

Introduction

Clustering images—grouping them by content without any labels—remains a fundamental challenge in computer vision. While humans can instantly distinguish between categories, machines often struggle with pixel-level noise. Traditional algorithms like K-means frequently fail on natural images because they rely on raw pixel distances, which do not capture semantic meaning. To address this, researchers have turned to deep learning to compress images into meaningful feature representations.

However, two major hurdles persist: the separation gap, where feature learning and clustering are treated as independent tasks, and the supervision trap, where high-performing models rely on pre-trained networks. This paper proposes a framework that learns discriminative features from scratch in a purely unsupervised manner by jointly optimizing for representation and clustering.

Methodology

The core of the proposed approach is a Deep Convolutional Autoencoder (DCA). The autoencoder learns to compress an image into a low-dimensional representation and subsequently reconstruct the original image. To ensure these representations are cluster-friendly, the authors introduce three key innovations:

  1. Inter-class Separation Constraint ($L_t$): Inspired by triplet loss, the model processes an Anchor image, a Positive image (an augmented version of the anchor), and a Negative image. The objective is to minimize the distance between the anchor and positive representations while maximizing the distance to the negative.
  2. Smart Negative Selection: To avoid selecting a negative image that might actually belong to the same class as the anchor, the authors developed an algorithm that selects a negative image only if its representation falls outside the distribution of the anchor's augmented variations.
  3. Entropy Minimization ($L_b$): To prevent "cluster collapse," where the model assigns most images to a few clusters, this constraint encourages confident and balanced assignments across all available clusters.
Architecture of the proposed Deep Convolutional Autoencoder (DCA) with triplet constraints for inter-class separation.

Experimental Results

The model was evaluated on standard benchmarks including MNIST, USPS, CIFAR-10, and a subset of ImageNet. The proposed method achieved state-of-the-art results, reaching 98.93% accuracy on MNIST and 97.63% on USPS. Notably, the model demonstrates strong generalization; unlike many competitors, it does not rely on pre-trained weights to perform well on complex datasets like CIFAR.

Ablation studies highlight the importance of the combined loss functions. As shown in the performance curves, the inclusion of the inter-class separation ($L_t$) and entropy minimization ($L_b$) terms significantly boosts accuracy compared to using standard clustering and reconstruction losses alone.

Clustering accuracy across epochs demonstrating the effectiveness of combining different loss components ($L_c, L_r, L_b, L_t$).

Conclusion

By jointly optimizing for feature representation and clustering through inter-class separation, this research provides a robust framework for unsupervised image grouping. The method effectively uses data augmentation as a proxy for labels, ensuring that similar images are pulled together while dissimilar ones are pushed apart. Future work will focus on refining the auxiliary distribution learning to further improve performance on highly complex, feature-rich datasets like the full ImageNet.

Summary updated

Abstract

Despite significant advances in clustering methods in recent years, the outcome of clustering of a natural image dataset is still unsatisfactory due to two important drawbacks. Firstly, clustering of images needs a good feature representation of an image and secondly, we need a robust method which can discriminate these features for making them belonging to different clusters such that intra-class variance is less and inter-class variance is high. Often these two aspects are dealt with independently and thus the features are not sufficient enough to partition the data meaningfully. In this paper, we propose a method where we discover these features required for the separation of the images using deep autoencoder. Our method learns the image representation features automatically for the purpose of clustering and also select a coherent image and an incoherent image simultaneously for a given image so that the feature representation learning can learn better discriminative features for grouping the similar images in a cluster and at the same time separating the dissimilar images across clusters. Experiment results show that our method produces significantly better result than the state-of-the-art methods and we also show that our method is more generalized across different dataset without using any pre-trained model like other existing methods.
BibTeX
@inproceedings{das2019deep,
  author = {Dipanjan Das and Ratul Ghosh and Brojeshwar Bhowmick},
  title = {{Deep Representation Learning Characterized by Inter-class Separation for Image Clustering}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2019},
  url = {https://openreview.net/forum?id=0i9MXDxOcKl},
}
Figure