In one dimension, sorting data is trivial. We can easily find the median, calculate quantiles, and identify outliers simply by looking at which points lie at the far edges of a number line. But when we move to multivariate spaces—such as high-dimensional text embeddings or neural network activations—this natural linear ordering vanishes. How do you define the "median" of a cloud of points in a 768-dimensional space?
Traditional approaches often rely on distance metrics like the Mahalanobis distance or assume the data follows a neat parametric distribution like a Gaussian bell curve. However, as Pierre Colombo et al. point out in Beyond Mahalanobis Distance for Textual OOD Detection, neural network representations frequently violate these Gaussian assumptions. Furthermore, inverting large covariance matrices introduces severe numerical instability and computational overhead.
To solve this, researchers leverage statistical data depth, a non-parametric framework that provides a center-outward ordering of multivariate data. Instead of sorting from left to right, data depth assigns a scalar value to each point representing its geometric centrality relative to the data cloud. The closer a point is to the "deepest" core of the distribution, the higher its depth score; the further it drifts toward the periphery, the lower its score.
The Geometry of Centrality
How do we compute a centrality score without assuming a specific probability distribution? Different depth functions use different geometric primitives to measure how "surrounded" a point is by the rest of the dataset.
Halfspace Depth
Imagine a query point inside a cloud of data. If you draw any arbitrary plane (or hyperplane in higher dimensions) through this point, it splits the space into two halfspaces. If the point is deep within the center of the data, any plane you draw will leave a substantial fraction of data points on both sides. If the point is an outlier on the edge, you can easily find a plane that leaves almost all the data on one side and the query point isolated on the other.
The halfspace depth is simply the minimum fraction of data points contained in any closed halfspace defined by a plane passing through the query point. Pierre Colombo et al. demonstrate in Toward Stronger Textual Attack Detectors that this distribution-agnostic measurement provides a highly reliable way to flag adversarial inputs without needing to model the complex underlying shape of text embeddings.
Lens Depth
Another elegant geometric approach is lens depth. To calculate it, you pick pairs of reference points from your dataset and draw spheres centered at each point such that their boundaries intersect. This creates a "lens" or an overlapping region between the two balls.
As Hai-Vy Nguyen et al. explain in Combining Statistical Depth and Fermat Distance for Uncertainty Quantification, a query point's lens depth is determined by how frequently it falls within these pairwise intersections. A point nestled in a dense, central cluster will naturally be swallowed up by many of these lenses, while a peripheral outlier will rarely land inside one.
Central Point (High Depth) Peripheral Point (Low Depth)
.---. .---.
/ \ / \
( * ) ( )
\ / \ /
'---' '---'
Swallowed by many Rarely falls inside
pairwise intersections intersecting lenses
By using these purely geometric or rank-based rules, data depth acts as a multivariate substitute for absolute values, as noted by Christophe Ley et al. in Univariate and multivariate symmetry: statistical inference and distributional aspects. It allows us to define multivariate medians and nested, robust "depth-trimmed regions" (Guillaume Staerman et al. in A Pseudo-Metric between Probability Distributions based on Depth-Trimmed Regions) without requiring any prior knowledge of the underlying probability distribution.
Why Data Depth Works for High-Dimensional Anomalies
The primary advantage of data depth is that it is distribution-agnostic. In modern machine learning, data complexity and dimensionality often trigger the curse of dimensionality, rendering traditional p-values and asymptotic distributions unreliable (Chengde Qian et al. in ByMI: Byzantine Machine Identification with False Discovery Rate Control).
When a model encounters an out-of-distribution (OOD) sample or an adversarial attack, these anomalous inputs often lie on the fringes of the expected feature manifold. Because data depth functions map complex high-dimensional geometry directly to a simple scalar rank based on centrality, they can easily flag these peripheral points (Romain Valla in Anomaly detection using data depth: multivariate case).
Furthermore, because it avoids parametric assumptions, it doesn't break down when data is skewed or multimodal. For example, Marine Picot et al. show in A Halfspace-Mass Depth-Based Method for Adversarial Attack Detection that measuring the centrality of an input relative to the training distribution maintains high reliability even as the complexity of the underlying classification task increases. Similarly, Parker Seegmiller et al. demonstrate in Statistical Depth for Ranking and Characterizing Transformer-Based Text Embeddings that data depth can effectively rank transformer-based text embeddings, identifying the most representative prompts for in-context learning and catching subtle distribution shifts in synthetic datasets.
Extending Depth Beyond Vector Spaces
The intuition of center-outward ranking is so powerful that it extends far beyond standard Euclidean vectors into complex mathematical structures:
- Functional Data: When dealing with continuous curves or time-series trajectories, the authors of Functional Isolation Forest show that recursive partitioning can be interpreted as a functional statistical depth measure, ranking how typical a trajectory is relative to a dataset.
- Graphs and Networks: In massive blockchain transaction networks, the authors of Data depth and core-based trend detection on blockchain transaction networks use multivariate data depth to enhance core decomposition, ranking nodes by their structural connectivity and transaction volume to identify stable, influential market actors.
- Permutations and Partial Orders: Even in non-vector spaces where traditional averaging is impossible, depth can find a consensus. Morgane Goibert et al. extend depth to discrete rankings and permutations in Statistical Depth Functions for Ranking Distributions: Definitions, Statistical Learning and Applications, while Hannah Blocher et al. adapt it to partial orders in Depth functions for partial orders with a descriptive analysis of machine learning algorithms, providing a way to define a median structure when multiple conflicting metrics make linear ranking impossible.
- Missing Data Imputation: Beyond detection, depth can be used for reconstruction. Julie Josse et al. leverage the global topological awareness of data depth in Nonparametric Imputation by Data Depth to fill in missing values without distorting the underlying shape of the data cloud.
Ultimately, statistical data depth provides a mathematically rigorous yet intuitively simple toolkit. By shifting our perspective from absolute coordinates to relative centrality, it allows us to navigate, filter, and understand high-dimensional spaces with the same clarity we enjoy in a simple one-dimensional line.
Key Foundational Work
Univariate and multivariate symmetry: statistical inference and distributional aspects (2010) — contributed 3 concept elements to this direction.
Depth-based runs tests for bivariate central symmetry (2012) — contributed 3 concept elements to this direction.