Angular triangle distance for ordinal metric learning

Imam Mustafa Kamal, Hyerim Bae, Ling Liu

TL;DR

This paper introduces a novel Angular Triangle Distance (ATD) and an Ordinal Triplet Network (OTN) to address the challenge of preserving the inherent order of ordinal data in deep metric learning (DML). Their method achieves more accurate and semantically meaningful embeddings for ordinal data compared to existing DML approaches.

What Problem Does This Paper Address?

Deep Metric Learning (DML) aims to learn a low-dimensional representation of data where similar items are close and dissimilar items are far apart. However, many real-world datasets have an ordinal nature, meaning their categories have a defined order (e.g., severity of a disease from mild to severe, product quality from low to high). Existing DML methods often treat these ordinal categories as simple nominal labels, failing to capture this crucial ordering. This can lead to embeddings where the semantic relationships between ordered categories are distorted or lost, hindering downstream tasks that rely on this inherent order.

Methods

The core of this work lies in two novel components:

  • Angular Triangle Distance (ATD): To overcome the limitations of traditional distance metrics (like Euclidean or cosine similarity) that don't inherently handle ordinality or satisfy metric properties, the authors propose ATD. It leverages the angular relationship between data representations in a low-dimensional space. ATD is designed to mathematically satisfy the properties of a distance metric (non-negativity, identity, symmetry, and triangle inequality).
  • Ordinal Triplet Network (OTN): This network is specifically designed to learn ordinal embeddings. It takes triplet inputs (representing three data points) and uses the ATD to project the ordinal relationships into an angular space. The network is trained using an ordinal regression approach to ensure that the learned embeddings accurately reflect the order of the original ordinal labels.

Datasets

The proposed method was evaluated on four public datasets with ordinal characteristics:

  • Breast Ultrasonic Images (Busi): Categorized into normal, benign, and malignant.
  • Finger Images: Used for counting fingers, with categories representing the number of fingers (0-5).
  • Face Age Recognition (FG-Net): Age groups are ordered (child, adolescent, adult, senior adult).
  • Adience: Another face age recognition dataset with ordered age categories.

The paper also mentions using several UCI machine learning repository datasets (Car, Nursery, Hayes-Roth, Balance) for comparison against conventional ordinal metric learning methods.

Key Insights and Ideas

  • Ordinality is Crucial: The paper strongly emphasizes that treating ordinal data as nominal in DML leads to suboptimal embeddings, even if accuracy metrics appear high. The inherent order matters for semantic representation.
  • Angles Over Distances: By using angular relationships (ATD), the method can better capture the relative ordering of categories, which is a key advantage over methods relying solely on Euclidean or cosine distances.
  • Triplet Structure for Ordinality: The triplet input structure, combined with ATD, allows the network to learn not just pairwise relationships but also the sequential order of ordinal categories.
  • Semantic Meaning vs. Accuracy: The authors highlight that high mapping accuracy (e.g., K-NN) doesn't always guarantee good semantic representation of ordinal relationships. Their method aims to achieve both.

Key Results

  • Improved Accuracy: The proposed OTD model consistently outperformed existing DML models (S-Net, T-Net, Q-Net, N-Net) in terms of mapping accuracy across the evaluated datasets, particularly in capturing ordinal relationships.
  • Semantic Preservation: Qualitative results (visualized through pairwise cosine distance matrices) demonstrate that OTD successfully preserves the ordinal relationships in the embedding space, unlike other DML models where these relationships tend to be scattered.
  • Competitive Performance: The method achieved competitive results against state-of-the-art ordinal regression techniques.
  • Dataset Specifics: The Finger dataset showed very high separability, leading to perfect accuracy for all models. The Busi dataset, with its similar features across categories, proved more challenging for triplet loss-based methods.

Limitations and Future Work

  • Computational Cost: The paper acknowledges that the triplet representation, while effective, can lead to higher computational costs, especially for large-scale datasets. This is identified as a direction for future research.
  • Generalizability of ATD: While ATD is shown to be a valid distance metric, its application to other machine learning methods beyond DML is suggested as a future avenue.
  • Hyperparameter Tuning: The paper mentions extensive experimentation to find optimal hyperparameters, implying that careful tuning is necessary for optimal performance.

Summary updated

Abstract

Deep metric learning (DML) aims to automatically construct task-specific distances or similarities between data, resulting in a low-dimensional representation. Several significant DML methods have been proposed. However, no approach guarantees the preservation of the ordinal nature of the original data in a low-dimensional space. Ordinal data are ubiquitous in real-world problems, such as the severity of symptoms in biomedical cases, production quality in manufacturing, rating level in businesses, and ageing level in face recognition. This study proposes a novel angular triangle distance (ATD) and ordinal triplet network (OTN) to obtain an accurate and meaningful embedding space representation for ordinal data. The ATD projects the ordinal relation of data in the angular space, whereas the OTN learns its ordinal projection. We also demonstrate that our new distance measure mathematically satisfies the distance metric properties. The proposed method was assessed using various benchmarks with an ordinal nature in both dependent and independent variables. Extensive experiments have been conducted, and the results show that our proposed method not only semantically preserves the ordinal nature but is also more accurate than existing DML models. Furthermore, our study demonstrates that our proposed method can effectively address ordinal regression tasks, producing competitive outcomes achieved by current state-of-the-art techniques.
BibTeX
@article{kamal2025angular,
  author = {Imam Mustafa Kamal and Hyerim Bae and Ling Liu},
  title = {{Angular triangle distance for ordinal metric learning}},
  journal = {Applied Intelligence},
  year = {2025},
  url = {https://openreview.net/forum?id=aSMYPYGYws},
}
Figure