CORE: Learning Consistent Ordinal REpresentations for Image Ordinal Estimation

Yiming Lei, Zilong Li, Yangyang Li, Junping Zhang, Hongming ShanarXiv 2023 · DOI · arXiv

Introduction: The Challenge of Ordinal Consistency

In computer vision, many classification tasks involve labels with a natural, intrinsic order. Examples include facial age estimation, disease progression stages, or image quality ratings. This field, known as Ordinal Regression (OR), differs from standard classification because the relationships between categories (e.g., age 20 is closer to 25 than to 60) are as important as the categories themselves.

While existing deep learning models for OR focus on optimizing the final prediction mapping, they often neglect the structure of the latent feature space. In many state-of-the-art models, the internal representations of samples do not necessarily reflect their ordinal relationships. For instance, the feature vector for a 60-year-old might be geometrically closer to that of a 20-year-old than a 55-year-old. This lack of "ordinal consistency" limits model robustness and interpretability.

CORE aligns the feature space geometry with the intrinsic order of labels.

Methodology: The CORE Regularizer

To address this, the authors propose CORE (Consistent Ordinal REpresentations), a "plug-and-play" regularizer designed to force the latent feature space to mirror the natural ordering of labels.

1. Ordinal Toset Distribution (OTD)

CORE treats a mini-batch of samples as a "totally ordered set" (toset). It calculates pairwise distances between labels to create a target probability distribution. Simultaneously, it computes distances between the corresponding feature embeddings. The goal is to align these two distributions.

2. Prototype-Constrained Optimization

Directly aligning features and labels is challenging due to the domain gap between images and numerical ranks. CORE formulates this as a constrained optimization problem using Convex Programming with Dual Decomposition. This approach derives an optimal feature embedding that respects an "ordinal prototype constraint," ensuring that samples with the same label cluster together while maintaining smooth transitions between adjacent ranks.

3. Seamless Integration

CORE is not a standalone architecture but a supplementary loss function ($L_{CORE}$). It can be integrated into existing deep OR models (such as SORD or POE) without altering their underlying structure, providing a significant performance boost with minimal overhead.

Experimental Results

The effectiveness of CORE was validated across eight datasets covering four distinct domains:

  • Facial Age Estimation: On the MORPH dataset, CORE reduced the Mean Absolute Error (MAE) from 2.41 to 2.13 when added to the Mean-Variance baseline.
  • Medical Imaging: In Diabetic Retinopathy classification, accuracy improved by up to 3.4%.
  • Historical Dating & Aesthetics: Consistent improvements were observed in estimating the decade of historical photos and ranking image quality (AADB dataset).

Key Insights: The 1D Ordinal Manifold

One of the most significant findings is the impact of CORE on the geometry of the feature space. Visualization via t-SNE reveals that while standard methods produce scattered or clumpy clusters, CORE forces features onto a nearly one-dimensional ordinal manifold. This curved line represents the learned "sequence" of the data, demonstrating that the model has internalized the ordinal nature of the task.

t-SNE visualizations showing the emergence of a 1D ordinal manifold.

Furthermore, Grad-CAM analysis shows that CORE-trained models focus on more semantically relevant features—such as skin texture for age estimation—rather than background noise, leading to better interpretability and grounding.

Conclusion

CORE provides a robust framework for ensuring that deep learning models respect the inherent structure of ordinal data. By aligning latent feature spaces with label-based tosets, it enhances both the accuracy and the internal logic of ordinal regression models across diverse applications.

Summary updated

Abstract

The goal of image ordinal estimation is to estimate the ordinal label of a given image with a convolutional neural network. Existing methods are mainly based on ordinal regression and particularly focus on modeling the ordinal mapping from the feature representation of the input to the ordinal label space. However, the manifold of the resultant feature representations does not maintain the intrinsic ordinal relations of interest, which hinders the effectiveness of the image ordinal estimation. Therefore, this paper proposes learning intrinsic Consistent Ordinal REpresentations (CORE) from ordinal relations residing in groundtruth labels while encouraging the feature representations to embody the ordinal low-dimensional manifold. First, we develop an ordinal totally ordered set (toset) distribution (OTD), which can (i) model the label embeddings to inherit ordinal information and measure distances between ordered labels of samples in a neighborhood, and (ii) model the feature embeddings to infer numerical magnitude with unknown ordinal information among the features of different samples. Second, through OTD, we convert the feature representations and labels into the same embedding space for better alignment, and then compute the Kullback Leibler (KL) divergence between the ordinal labels and feature representations to endow the latent space with consistent ordinal relations. Third, we optimize the KL divergence through ordinal prototype-constrained convex programming with dual decomposition; our theoretical analysis shows that we can obtain the optimal solutions via gradient backpropagation. Extensive experimental results demonstrate that the proposed CORE can accurately construct an ordinal latent space and significantly enhance existing deep ordinal regression methods to achieve better results.
BibTeX
@misc{lei2023core,
  author = {Yiming Lei and Zilong Li and Yangyang Li and Junping Zhang and Hongming Shan},
  title = {{CORE: Learning Consistent Ordinal REpresentations for Image Ordinal Estimation}},
  year = {2023},
  eprint = {2301.06122},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2301.06122},
}

Loading related content...

Figure