Learning with Negative Eigenvalues in Non-Metric Data

You are reading version 1 of this direction, frozen 2026-10-04. The latest version is version 2.

Version 1 of 228 papers51 conceptsfrozen 2026-10-04

Most machine learning algorithms assume that data lives in a "well-behaved" Euclidean space where the triangle inequality always holds. However, real-world relationships—such as human perceptual judgments, protein sequence similarities, or Jaccard distances—frequently violate these geometric rules. When we represent these non-metric relationships as a similarity matrix, the mathematical consequence is the appearance of negative eigenvalues.

Traditionally, researchers viewed these negative components as noise or measurement errors to be "fixed" by zeroing them out. A growing body of research suggests the opposite: these negative eigenvalues encode essential structural information that Euclidean geometry simply cannot capture.

The Geometry of Violation

To understand why negative eigenvalues matter, we must first understand what they represent. In a standard Euclidean space, if point A is close to B, and B is close to C, then A cannot be too far from C. This is the triangle inequality.

But as Julian Laub demonstrates in Feature Discovery in Non-Metric Pairwise Data, many datasets are inherently non-metric. For example, in language, a specific word might be "close" to a general category, but two specific words in that same category might be very "far" from each other. This "non-metricity" is not a mistake; it is a signal of latent features like word specificity or, in handwriting analysis, stroke weight.

When we perform an eigendecomposition on a similarity matrix, the positive eigenvalues represent dimensions that follow Euclidean rules. The negative eigenvalues represent "pseudo-Euclidean" dimensions. As Mark Blumstein explains in Multi-Dimensional Scaling on Groups, these components allow us to reconstruct distances exactly by accounting for the ways the data "breaks" Euclidean geometry.

The Cost of "Fixing" Data

The standard practice in the field has been to force non-metric data into a metric-compliant format. Common techniques include "clipping" (setting negative eigenvalues to zero) or "flipping" (taking their absolute value).

However, Frank-Michael Schleif and Peter Tiňo argue in their survey of indefinite proximity spaces that this "correction" is destructive. By forcing the data into a positive semi-definite (PSD) cone, we discard the very structural violations that define the domain. Stefanos Zafeiriou shows in Subspace Learning in Krein Spaces that the "negative subspace" often contains highly discriminative information. In tasks like face recognition, ignoring these dimensions leads to a significant loss in accuracy.

Embracing Indefinite Spaces

Instead of fixing the data, modern approaches adapt the math to fit the data. This involves moving from Hilbert spaces (which are strictly positive) to Krein spaces or pseudo-Euclidean spaces.

In these spaces, the "distance" between two points is calculated using a symmetric bilinear form rather than a standard inner product. Essentially, some dimensions contribute positively to the distance, while others contribute negatively.

Recent breakthroughs have made this approach practical:

  • Neuc-MDS: Chengyuan Deng and colleagues introduced Neuc-MDS, which uses bilinear forms to resolve the "dimensionality paradox"—a phenomenon where adding dimensions in classical MDS can actually increase error because it ignores the negative spectrum.
  • Neural Similarity Encoders (SimEc): Franziska Horn developed SimEc, a neural architecture that learns to map features directly to these indefinite embeddings, allowing for out-of-sample predictions without assuming the similarity is metric.
  • Complex Embeddings: In Multi-perspective embedding for non-metric time series, Maximilian Münch suggests preserving negative eigenvalue information in the imaginary components of a complex-valued vector space.

Why It Works

The intuition is that negative eigenvalues act as a "correction factor" for Euclidean over-simplification. If you try to flatten a globe onto a 2D map, you inevitably get distortion. Standard MDS tries to minimize this distortion by stretching the map. Indefinite embeddings, by contrast, acknowledge that the "surface" of the data is fundamentally curved or non-Euclidean.

By retaining the negative spectrum, algorithms like Non-Metric Locality-Sensitive Hashing by Yadong MU can capture human-like perceptual similarities that violate the triangle inequality, leading to more accurate retrieval and classification in complex, real-world domains.

Papers in this version

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 24 papers
  2. Version 1 · 2026-10-04 · 28 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_negative_eigenvalue_information_loss_33bacada,
  title = {negative eigenvalue information loss},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/negative-eigenvalue-information-loss/rdr_33bacada346643c4a9a1353479d2f2ee},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_learning_with_negative_eigenvalues_in_no_33bacada_v1,
  title = {Learning with Negative Eigenvalues in Non-Metric Data},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/learning-with-negative-eigenvalues-in-non-metric-data/rdr_33bacada346643c4a9a1353479d2f2ee/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}