Learning Embeddings for Image Clustering: An Empirical Study of Triplet Loss Approaches

Kalun Ho, Janis Keuper, Franz-Josef Pfreundt, Margret Keuper

Abstract

Unsupervised image clustering depends heavily on the quality of learned feature embeddings. While Triplet Loss is a standard objective for learning these representations, its focus on relative distances often hinders performance in graph-based clustering scenarios that require stable absolute distance thresholds. This paper presents an empirical study of Triplet Loss variants, proposing a simplified version that explicitly constrains intra- and inter-cluster distances. Evaluated on the CIFAR-10 dataset, our proposed loss enables effective graph-based clustering and demonstrates significant robustness to label noise, outperforming existing methods in challenging training conditions.

Introduction

Clustering images into meaningful categories is a fundamental task in computer vision. Modern approaches leverage Convolutional Neural Networks (CNNs) to map images into a high-dimensional embedding space where similar images are close together. A common training objective for these embeddings is the Triplet Loss, which optimizes the relative distance between an anchor, a positive sample, and a negative sample. However, this focus on relative positioning can be problematic for certain clustering paradigms, particularly those that rely on global distance thresholds.

The Challenge of Absolute Distances

Standard Triplet Loss ($L_{triplet\_1}$) ensures that a positive sample is closer to an anchor than a negative sample by at least a certain margin. While this is sufficient for centroid-based methods like k-means, it fails to stabilize the absolute distances across different classes. In graph-based clustering, such as correlation clustering (or the minimum cost multicut problem), decisions to join or split nodes depend on a global distance threshold. Without stable absolute distances, a single threshold cannot effectively separate clusters across the entire dataset, as the intra-class variance can differ wildly between categories.

Overview of the proposed image clustering pipeline, from training on triplets to graph-based evaluation.

Proposed Methodology

The researchers evaluated three Triplet Loss variants to address these limitations:

  1. Standard Triplet Loss ($L_{triplet\_1}$): Focuses purely on relative distance margins.
  2. Improved Triplet Loss ($L_{triplet\_2}$): Introduces a constraint to limit the maximum distance between positive pairs.
  3. Proposed Triplet Loss ($L_{triplet\_3}$): A simplified variant using absolute distance margins. It pushes negative pairs beyond a distance $\alpha$ and pulls positive pairs within a distance $\beta$.

By employing $L_{triplet\_3}$, a fixed distance threshold $\tau = \sqrt{(\alpha + \beta)/2}$ can be mathematically derived, providing a consistent metric for graph-based clustering. The study utilizes a modified AlexNet architecture to produce 32-dimensional embeddings for the CIFAR-10 dataset, which contains 60,000 images across 10 classes.

Experimental Results

The performance of these loss functions was evaluated using both k-means and the minimum cost multicut approach.

Comparison of clustering accuracy for Correlational Clustering (a) and K-Means (b) across the three Triplet Loss variants.

As shown in the results, the proposed $L_{triplet\_3}$ (Method 3) achieved high accuracy across both clustering methods, reaching 80.5% for multicut and 81.7% for k-means. Notably, while k-means is somewhat resilient to distance scaling issues, the graph-based multicut approach showed significant improvement with the proposed loss compared to the standard variant. The multicut method also demonstrated an ability to discover hierarchical sub-structures, such as separating specific visual sub-types within a class, which k-means (with a fixed $k=10$) cannot do.

Furthermore, the proposed loss proved highly robust to label noise. In experiments where 20% of the training triplets contained incorrect labels, $L_{triplet\_3}$ maintained its performance, outperforming previous state-of-the-art variants by over 10% in multicut accuracy. This stability makes it particularly suitable for real-world scenarios where perfectly labeled training data is scarce.

Conclusion

This study highlights the importance of absolute distance stability in learned embeddings for graph-based clustering. By moving from relative to absolute distance margins in the Triplet Loss objective, we can achieve more robust and flexible clustering performance. While graph-based methods like multicut offer greater flexibility in discovering data structure, their computational complexity remains a challenge for future work to address at scale.

Summary updated

BibTeX
@inproceedings{ho2020learning,
  author = {Kalun Ho and Janis Keuper and Franz-Josef Pfreundt and Margret Keuper},
  title = {{Learning Embeddings for Image Clustering: An Empirical Study of Triplet Loss Approaches}},
  booktitle = {ICPR},
  year = {2020},
  url = {https://openreview.net/forum?id=8l8WWijWup},
}
Figure