Deep Repulsive Clustering of Ordered Data Based on Order-Identity Decomposition

Seon-Ho Lee, Chang-Su KimICLR 2021 · OpenReview

TL;DR

This paper introduces Deep Repulsive Clustering (DRC), a framework that improves rank estimation (such as predicting age or aesthetic scores) by separating "identity" features from "order" features. By grouping similar objects into distinct clusters using a novel repulsive loss, the system estimates an object's rank more accurately by comparing it only to the most relevant reference samples.

The Challenge of Ordered Data

In many machine learning tasks, we deal with ordered data—data where classes have a natural sequence, such as age (18 < 19 < 20) or aesthetic quality. A recent trend called "order learning" suggests that it is easier for a model to predict the relative relationship between two objects (e.g., "Who is older?") than to predict an absolute value ("How old is this person?").

However, comparing objects is difficult when they possess vastly different "identity" characteristics. For example, in facial age estimation, it is often easier to compare the ages of two people of the same gender or ethnicity than to compare across entirely different demographic groups.

Examples of ordered data (age progression) across diverse identities.

Previous attempts to group ordered data into "chains" often resulted in messy, overlapping clusters that failed to capture meaningful properties. The authors aim to solve this by explicitly decomposing data into rank-related and rank-unrelated features.

Proposed Method: ORID and DRC

The authors propose a two-part approach to tackle this disentanglement and clustering problem: the ORID Network and the DRC Algorithm.

1. Order-Identity Decomposition (ORID)

ORID is a multi-task neural network consisting of an encoder, a decoder, a comparator, and a discriminator.

  • The Encoder splits the input into two distinct feature vectors: $h_{or}$ (order-related) and $h_{id}$ (identity-related).
  • The Comparator ensures $h_{or}$ contains the ranking information by training on pairwise comparisons.
  • The Decoder and Discriminator (GAN) ensure that $h_{id}$ captures the remaining structural information needed to reconstruct the original image. This effectively "bleeds" the rank information out of the identity feature, leaving a pure representation of the subject's underlying characteristics.

2. Deep Repulsive Clustering (DRC)

Once identity features are extracted, the model groups them. Unlike standard k-means clustering, DRC uses a repulsive term in its objective function. This term explicitly pushes cluster centroids away from each other on a unit sphere. This ensures that clusters (e.g., "Males" vs. "Females" or "Landscapes" vs. "Portraits") are clearly separated and distinct, preventing the "crowding" often seen in high-dimensional spaces.

3. Rank Estimation

To predict the rank of a new test sample, the model first assigns it to one of these well-defined clusters. It then uses a Maximum A Posteriori (MAP) estimation rule to compare the test sample against reference images within that same cluster. This leads to a much more accurate, "apples-to-apples" comparison.

Datasets and Evaluation

The model was rigorously evaluated on three distinct types of ordered data:

  • Facial Age Estimation: Evaluated on MORPH II (a large dataset of 55,000 images with gender and race labels) and a custom Balanced Dataset combining MORPH II, AFAD, and UTK to mitigate ethnic and gender bias.
  • Aesthetic Score Regression: Evaluated on the AADB (Aesthetics and Attribute Database), containing 10,000 photographs with aesthetic scores ranging from 0 to 1.
  • Historical Image Classification: Evaluated on HCI (Historical Color Image), a dataset of 1,050 images categorized by the decade they were taken (1930s–1970s).

Key Insights and Results

The core "aha moment" of this research is that identity features (like gender in age estimation or style in aesthetics) act as noise when trying to learn order. By explicitly pushing these features into a separate vector, the model can focus on the "pure" signal of the rank. Interestingly, the model discovered meaningful clusters (like gender and race) entirely on its own, without being given those labels during the clustering phase.

This approach yields impressive empirical results:

  • State-of-the-Art (SOTA) Performance: The DRC-ORID method achieved SOTA results on the MORPH II dataset, reaching a Mean Absolute Error (MAE) of 2.26, outperforming previous methods like BridgeNet and OL.
  • Aesthetic Regression: On the AADB dataset, the model achieved an MAE of 0.1056, significantly better than standard regression models.
  • Qualitative Success: t-SNE visualizations demonstrated that while previous methods had overlapping clusters, DRC produces clearly defined, separated clusters that correlate strongly with human-understandable categories.

Limitations and Future Work

While powerful, the framework has limitations. The performance can be sensitive to the choice of $k$ (the number of clusters). If $k$ is too high for a small dataset, clusters may contain too few samples to train the comparator effectively.

Furthermore, the authors explicitly note ethical considerations: because the model clusters based on identity features like race and gender, there is a risk of these features being misused for discriminatory purposes in facial recognition systems. They recommend the tool be used strictly for research purposes.

Looking forward, the authors suggest that this repulsive clustering framework could be applied to other complex ranking problems beyond computer vision, such as recommendation systems or document ranking.

Summary updated

Abstract

We propose the deep repulsive clustering (DRC) algorithm of ordered data for effective order learning. First, we develop the order-identity decomposition (ORID) network to divide the information of an object instance into an order-related feature and an identity feature. Then, we group object instances into clusters according to their identity features using a repulsive term. Moreover, we estimate the rank of a test instance, by comparing it with references within the same cluster. Experimental results on facial age estimation, aesthetic score regression, and historical color image classification show that the proposed algorithm can cluster ordered data effectively and also yield excellent rank estimation performance.
BibTeX
@inproceedings{lee2021deep,
  author = {Seon-Ho Lee and Chang-Su Kim},
  title = {{Deep Repulsive Clustering of Ordered Data Based on Order-Identity Decomposition}},
  booktitle = {International Conference on Learning Representations},
  year = {2021},
  url = {https://openreview.net/forum?id=Yz-XtK5RBxB},
}

Loading related content...

Figure