Most methods that recover the shape of high-dimensional data work locally first. Around each point they fit a small flat piece, a tangent plane, and then glue the pieces into one global map. This breaks exactly where real data is most interesting: at points where two sheets cross, where branches meet, or where a surface pinches to a point. There the data does not look like any single plane, so the local fits fail and the gluing step spreads the error across the whole map. Language-model token embeddings, molecular conformation spaces, and branching cell trajectories all contain such points. Tools exist that detect them quickly, but none that reconstruct the geometry through them. Inverse-problem theory has the matching gap. Recovering a space from partial measurements is proven possible in principle, sometimes with good stability, but no algorithm turns the proofs into a computation on finite samples.
We propose to replace the tangent plane with the tangent cone, meaning the simplest exact model of what a space looks like under a microscope at a given point. At a smooth point the tangent cone is a plane. At a crossing it is two planes through a common point. At a branch it is several half-planes glued along an edge, and at a pinch it is a cone. The method fits each neighborhood to a small library of such models, uses the fitted models to decide which nearby points are truly connected inside the space, and stitches the local pieces into a discrete metric space, a set of points with distances measured along the space. The fitting also gives a number: the radius at which each local model stops fitting. We feed that radius into a known Lipschitz stability bound for reconstructing spaces from distance data, which produces an a-posteriori error certificate for the global reconstruction.
Core Claim
The claim is that first-order local models of stratified spaces are enough to carry local-to-global reconstruction through singularities. A stratified space is one made of smooth pieces, called strata, of possibly different dimensions, joined along lower-dimensional seams. These models are enough without second-order curvature estimates, and the number of parameters per neighborhood grows linearly in intrinsic dimension per sheet rather than quadratically. The scale at which these local models hold also plays the role of the local-embedding scale that appears in stability theorems for travel-time inversion, which turns a non-constructive stability result into a computable error bound.
Three things would be new. The first is reconstruction, not just detection, through singular points. The second is identifying the type of singularity as a byproduct of model selection. The third is an error certificate that comes from inverse-problem stability theory and not from heuristics.
Why the Distance Route, Not the Operator Route
The inverse-problems literature offers two ways to recover a space from partial data. The operator route measures something like the Dirichlet-to-Neumann map, which records how the boundary responds to applied voltages or heat. The Dirichlet-to-Neumann map for complete Riemannian manifolds with boundary proves that this map on any small boundary patch determines a real-analytic manifold up to isometry. The proof goes through analytic continuation, which gives no algorithm and does not survive corners. Metric tensor estimates, geometric convergence, and inverse boundary problems gets conditional stability, but only under a priori curvature and injectivity bounds that are unavailable for blind data. Once the boundary is merely Lipschitz, Conditional stability estimation for an inverse boundary problem with non-smooth boundary in R^3 shows stability degrades to a double-logarithmic rate in the noise level. That is effectively uncomputable. These papers constrain the design: they tell us not to route reconstruction of singular spaces through operators that rely on analytic continuation.
The distance route behaves differently. Lipschitz Stability of Travel Time Data shows that length spaces can be recovered from travel times measured from a closed subset $R$, with Lipschitz rather than logarithmic stability. Length spaces are spaces where distance means length of the shortest path inside the space, and they include metric trees and domains with holes. The recovery holds under a backward local isometric embeddability condition at scale $\epsilon$:
$$d_{GH}(X_1, X_2) \le \left(\frac{2D}{\epsilon} + 1\right) d_H\big(D_R(X_1), D_R(X_2)\big),$$
Here $d_{GH}$ is the Gromov–Hausdorff distance between two spaces $X_1, X_2$, which measures how far they are from being isometric. $D_R(X)$ is the set of distance functions from points of $X$ to the measurement set $R$, $d_H$ is the Hausdorff distance between two such sets, and $D$ bounds the diameter. The paper is non-constructive and leaves verification of the condition open. This proposal supplies the missing algorithm and an empirical estimate of $\epsilon$.
We should be explicit about one uncertainty. We read the embeddability condition as "neighborhoods of radius $\epsilon$ are isometric to their model," and we use the tangent-cone fitting radius as an estimate of $\epsilon$. Whether this matches the paper's exact definition has to be checked against the theorem before the certificate is called rigorous. Until then it is a calibrated heuristic, and the experiments test whether it tracks true error.
Research Program
The pipeline has three stages.
Local cone fitting. For each sample $x$ we take neighbors at several radii and fit models from a small library: one $d$-plane; $m$ $d$-planes through a point; $m$ half-planes sharing a $(d-1)$-dimensional edge; and a cone over a circle. Each sheet is fit by local principal components on a subset of neighbors. Points are assigned to sheets by an alternating, subspace-clustering-style procedure, and the model is chosen with a penalized likelihood. A model with $m$ sheets of dimension $d$ costs on the order of $m \cdot d \cdot D$ parameters in ambient dimension $D$. That is linear in $d$, unlike the $O(d^2)$ local Hessian fits that Curvature Enhanced Data Augmentation for Regression needs and that become underdetermined as $d$ grows.
The largest radius at which the chosen model passes a goodness-of-fit test is recorded as $\hat\epsilon(x)$. For the test we reuse the local measure-comparison idea of HADES, which compares neighborhoods against a uniform disk using a kernel two-sample statistic. We extend the comparison from a single disk to the fitted union of sheets. This fitting step does two things: it reconstructs local geometry, and it classifies singularity type. That classification is exactly what HADES and TOAST leave open.
Model-consistent connectivity. Neighborhood graphs fail near singularities because they add edges between points that are close in the ambient space but far apart inside the space. Examples are two sheets that come close without meeting, or the far side of a pinch. We keep an edge between two points only if both endpoints' fitted models place them on a common sheet, or on sheets that meet inside both models. We then compute shortest-path distances from a landmark set $R$, chosen by farthest-first traversal as in Intrinsic Dimension Adaptive Partitioning for Kernel Methods. These distances are the empirical travel-time data.
Stratum-wise stitching and certificate. Within each stratum we align local sheet coordinates using the global alignment step of Local Tangent Space Alignment. That method stitches local flat pieces by solving one eigenproblem, but it collapses where the local Jacobian is rank-deficient. We give it only sheets on which it is well defined, and we tie strata together at the singular seams found in the first stage. Bokor et al. reconstruct only one-dimensional graphs (Stratified Space Learning: Reconstructing Embedded Graphs). Our version handles strata of any dimension and needs no nonlinear least-squares initialization, because the seams come from the cone fits.
For the certificate, we estimate the travel-time perturbation $\hat\delta$ by bootstrap resampling of the point cloud. We then report $(2\hat D/\hat\epsilon_{\min} + 1)\hat\delta$, where $\hat\epsilon_{\min}$ is a lower quantile of $\hat\epsilon(x)$ and $\hat D$ is the estimated diameter. The paper's $\epsilon$-truncated variant of the bound, which avoids the diameter factor, is the natural refinement if $\hat D/\hat\epsilon$ turns out large.
We will also analyze sample complexity. The aim is to show that, at fixed sheet count $m$, the number of neighbors needed for correct model selection grows linearly in $d$ and $m$. The global sample size still grows exponentially in $d$, which no method escapes, as Sample Complexity of Testing the Manifold Hypothesis shows. Our claim is limited to removing the extra local penalty that second-order methods add.
Baselines
The naive reference point is a plain neighborhood graph with shortest-path distances, in the style of Isomap, with no singularity handling. It is the "do nothing" version of our second stage, and it shows how much damage singularities actually cause to metric recovery. It is run on the same samples and landmarks, with neighborhood size tuned on held-out synthetic shapes using the same budget we give our method. We count our method as beating it only if geodesic distortion near singular seams drops while distortion on smooth regions stays the same.
The strongest existing competitor for coordinate reconstruction is LTSA (Principal Manifolds and Nonlinear Dimension Reduction via Local Tangent Space Alignment). It is the most accurate first-order local-to-global stitcher in the file and the method our third stage generalizes, so any gain must be measured against it directly. It gets the same neighborhood-size search. We found no reported result on Lacuna for LTSA reconstruction error on singular spaces; its paper reports qualitative unrolling only. We will therefore run it ourselves.
For one-dimensional stratified data, the strongest competitor is Bokor et al.'s stratified graph learning (Stratified Space Learning), evaluated on the bow-tie graph they introduced. We found no reported result on Lacuna for quantitative vertex-placement error on that benchmark, so we will report vertex error and edge recovery for both methods.
For singular-point identification, the strongest competitor is HADES. It reports AUC of at least 0.89 for singularity detection on synthetic singular shapes of intrinsic dimension 1 through 5, and singularity identification on cyclo-octane in about 5 seconds on a laptop (HADES). Our detection must match that AUC within noise at comparable runtime. Our method does more than HADES does, so the goal is not to beat it at detection; the goal is to show that the extra work of reconstruction does not cost detection accuracy.
The key ablation is detect-then-cut. We remove points flagged by HADES, run LTSA separately on each connected piece, and reconnect the pieces through the removed points by nearest-neighbor matching. This isolates the claim that singular neighborhoods must be modeled rather than excised. A second ablation keeps cone fitting but disables edge pruning, which isolates the contribution of model-consistent connectivity to metric accuracy.
Why not just…?
Why not just use neighborhood-graph geodesics, which already recover a length space? This is the most serious threat. At a true crossing, the shortest path through the junction is a real path inside the space, so plain graph geodesics may already be accurate there. Our method has to win elsewhere: at near-contacts and pinches, where ambient-close points are falsely joined, and in providing stratification, type labels, and a certificate that geodesics alone do not give. The experiments separate these cases by shape. If plain geodesics match our distortion on every family, including pinched and near-tangent ones, we will conclude the cone-fitting step is not needed for the metric. We would then report it only as a classification and certification tool.
Why not just detect singularities with HADES and run standard manifold learning on what remains? The detect-then-cut ablation tests exactly this. Singular regions are dense rather than sparse, as HADES itself notes. Removing them splits the space into pieces whose relative placement is lost. On cyclo-octane, cutting along the two intersection circles separates the sphere from the Klein bottle, and nothing in the remaining data says how they reattach. We expect the cut version to have large distortion for distances that cross seams. If careful reattachment recovers those distances, the simpler pipeline wins and we will say so.
Why not just use diffusion maps or a graph Laplacian embedding? Diffusion maps, spectral clustering and reaction coordinates of dynamical systems shows these converge to smooth differential operators only under Riemannian smoothness. The sharp rates in Error Estimates for Spectral Convergence of the Graph Laplacian assume no boundary and no singularities. Random walks also concentrate on higher-dimensional strata, so a single spectral embedding misrepresents lower-dimensional pieces. We include diffusion maps as a secondary baseline on the mixed-dimension shapes, where we expect this failure to be measurable.
Why not just fit second-order curvature models locally? Curvature Enhanced Data Augmentation for Regression shows first-order fits cut corners on curved data. But its Hessian fits need neighborhoods that grow quadratically in $d$, and they are undefined at singular points. Our sweep over intrinsic dimension tests whether first-order cones lose accuracy on curved sheets relative to a second-order variant. Any loss at small $d$ should be weighed against a second-order method's sample needs at $d \ge 4$.
Proposed Experiments
The main testbed is synthetic: intersecting planes, branching half-planes, and pinched spaces of intrinsic dimension 1 through 5, embedded in ambient dimensions up to 100 with Gaussian noise. This follows the families used by HADES; we regenerate them, since the file does not confirm a released dataset. To these we add the pinched torus from TOAST and the bow-tie graph from Bokor et al. Ground-truth intrinsic distances are computable for all of them.
The metrics are maximum and mean geodesic distortion over all pairs. Maximum distortion upper-bounds twice the Gromov–Hausdorff error for the identity correspondence. We separately report distortion for pairs whose true geodesic crosses a seam. Further metrics are adjusted Rand index for stratum labels, singularity-type accuracy against the generating model, detection AUC, and wall-clock time.
For each shape family we vary sample size and $d$ and record the sample size needed for correct model selection. This tests the linear-in-$d$ local requirement against the quadratic requirement of second-order fits.
Cyclo-octane serves as the real-geometry check. Its structure is known: a sphere and a Klein bottle meeting along two circles. We test whether the method recovers two two-dimensional strata, one-dimensional seams that form two circles, and the right gluing. We compare runtime against the 5-second HADES figure and the multi-hour persistent-homology figure HADES quotes.
As an application probe without ground-truth metric, we run singularity typing on GPT-2 token embeddings. We ask whether the 66 tokens that fail the manifold test in Token Embeddings Violate the Manifold Hypothesis are mainly classified as multi-sheet junctions, which would be consistent with the polysemy explanation proposed there, or as cones and dimension jumps. This is descriptive, not a performance claim.
Two checks decide the project. The first is whether model-consistent connectivity reduces seam-crossing distortion relative to both plain geodesics and detect-then-cut on pinched and near-tangent shapes, while matching them on smooth regions. If it does not, the central mechanism fails. The second is whether the certificate tracks reality: across noise levels and sample sizes, the bound must stay above the measured distortion and rank configurations in the same order (Spearman correlation reported). A certificate that is valid but loose by orders of magnitude, or that sometimes falls below the true error, would show that the fitting radius is not a usable stand-in for the embeddability scale in the travel-time theorem. That outcome is informative in its own right about how far the stability theory can be made constructive.
References
- The Dirichlet-to-Neumann map for complete Riemannian manifolds with boundary
- Metric tensor estimates, geometric convergence, and inverse boundary problems
- Conditional stability estimation for an inverse boundary problem with non-smooth boundary in $\mathcal {R}^3$
- Lipschitz Stability of Travel Time Data
- Curvature Enhanced Data Augmentation for Regression
- HADES: Fast Singularity Detection with Local Measure Comparison
- TOAST: Topological Algorithm for Singularity Tracking
- Intrinsic Dimension Adaptive Partitioning for Kernel Methods
- Principal Manifolds and Nonlinear Dimension Reduction via Local Tangent Space Alignment
- Stratified Space Learning: Reconstructing Embedded Graphs
- Sample Complexity of Testing the Manifold Hypothesis
- Diffusion maps, spectral clustering and reaction coordinates of dynamical systems
- Error Estimates for Spectral Convergence of the Graph Laplacian on Random Geometric Graphs Toward the Laplace-Beltrami Operator
- Token Embeddings Violate the Manifold Hypothesis