Modeling Non-Uniform Distances in Ordinal Data

This direction is split into other directions: metric scale validity. Its last version stays readable and citable here.

Version 1 of 153 papers66 conceptsfrozen 2026-10-04

In machine learning and statistics, we frequently encounter data that is ordered but not measured. A movie rating might be 1 to 5 stars; a medical diagnosis might range from "benign" to "malignant"; a survey might ask if you "strongly disagree" or "strongly agree."

The standard reflex is to treat these labels as integers: $1, 2, 3, 4, 5$. By doing so, we implicitly assume equidistance—that the "distance" between 1 and 2 stars is exactly the same as the distance between 4 and 5 stars. However, as David M. Howcroft and Verena Rieser argue in What happens if you treat ordinal ratings as interval data?, this assumption is a "statistical sin" that weakens the power of our evaluations and distorts our models.

The Fallacy of the Average

The most common consequence of the equidistance assumption is the practice of averaging ordinal scores. If one person rates a story a "1" and another a "5," we say the average is "3." But as Kawin Ethayarajh and Dan Jurafsky demonstrate in The Authenticity Gap in Human Evaluation, this is mathematically unsound.

Averaging assumes a cardinal utility—that the psychological distance between points is uniform. In reality, human perception is non-linear. The effort required for a writer to move a reader from "mediocre" to "good" might be far greater than the effort to move them from "good" to "great." When we average these scores, we risk producing rankings that actually reverse true human preferences. This problem extends to Information Retrieval; Hui Fang and colleagues note in their report on ICTIR’17 that metrics like Average Precision do not reside on an interval scale, making their arithmetic means questionable.

Latent Thresholds and Learned Distances

If we cannot trust fixed integers, how should we model order? The research direction shifts toward treating ordinal categories as intervals on a latent continuous scale.

Instead of a single point, a category like "Moderate Pain" represents a range of underlying intensity. Koby Crammer and Yoram Singer pioneered this with PRank in Pranking with Ranking, which uses a perceptron-based approach to learn multiple parallel thresholds. Rather than predicting a number, the model predicts where an input falls relative to these learned boundaries.

This allows the model to discover that some categories are "closer" than others. For example, in medical imaging, Xiaofeng Liu and Xu Han use Wasserstein distance to account for the fact that the transition from "healthy" to "early-stage disease" is often more significant than moving between two advanced stages. By learning these flexible margins, models like CLOC (Contrastive Learning for Ordinal Classification) can prioritize critical decision boundaries, such as the threshold between benign and malignant tumors.

Beyond Regression and Classification

Ordinal data sits in a "middle ground" that standard tools struggle to capture:

  • Regression assumes the distances are fixed and known (usually 1.0).
  • Classification assumes the categories are independent, ignoring the fact that "Grade 2" is closer to "Grade 3" than to "Grade 5."

To bridge this, researchers have developed specialized algorithms. Yiqun Zhang and Yiu-ming Cheung proposed Distance Learning-based Clustering (DLC) in An Ordinal Data Clustering Algorithm with Automated Distance Learning, which automatically learns the "true" distances between categories during the clustering process. Similarly, Enrique Amigó and colleagues introduced the Closeness Evaluation Measure (CEM) in An Effectiveness Metric for Ordinal Classification, which uses information theory to determine proximity based on the actual distribution of the data rather than arbitrary numerical assignments.

Why It Works: Preserving Information

The intuition behind this research is information preservation. When we binarize ordinal data (e.g., "High" vs "Low") or force it into a linear regression, we throw away the nuanced structure of the data.

As Olivier Gouvert and colleagues show in Ordinal Non-negative Matrix Factorization for Recommendation, preserving the ordinal nature of user ratings leads to significantly better recommendations. By acknowledging that the "gap" between ranks is a variable to be learned rather than a constant to be assumed, these methods align the mathematics of the model with the messy, non-linear reality of human judgment and biological processes.

Papers in this version

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_modeling_non_uniform_distances_in_ordina_e28260d6,
  title = {Modeling Non-Uniform Distances in Ordinal Data},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/modeling-non-uniform-distances-in-ordinal-data/rdr_e28260d6c6984dffad8eed27fee53dcf},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 1, frozen text):

@misc{lacuna_modeling_non_uniform_distances_in_ordina_e28260d6_v1,
  title = {Modeling Non-Uniform Distances in Ordinal Data},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/modeling-non-uniform-distances-in-ordinal-data/rdr_e28260d6c6984dffad8eed27fee53dcf/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}