Distance Preserving Latent Representations

14 concepts9 papers1 proposal

When deep neural networks learn representations of the world, they typically map data into a high-dimensional latent space where proximity signifies semantic similarity. For instance, a language model might place the words "apartment" and "condo" close together because they appear in similar textual contexts.

However, when dealing with physical reality—such as geographic coordinates, urban maps, or the pixel layouts of images—this purely semantic arrangement breaks down. If a model relies solely on data features like human mobility or textual descriptions, two physically adjacent neighborhoods might be mapped to completely opposite sides of the latent space simply because one has high traffic and the other is quiet. This violates a fundamental principle of geography: spatial autocorrelation, or the idea that near things are more related than distant things.

To bridge this gap, a growing body of research focuses on distance-preserving latent representations. The core intuition is simple: structure the mathematics of the latent space so that the distance between two vector embeddings monotonically reflects the physical or geometric distance between the objects they represent.

The Problem with Standard Embeddings

Why do standard deep learning architectures struggle with space? Consider a standard contrastive learning framework. It typically treats pairs of data points as either binary matches (positives) or non-matches (negatives). If a model is trying to predict the location of an image or a trajectory path, a wrong prediction that is 5 miles away is penalized just as harshly as a wrong prediction that is 5,000 miles away.

As Zhang et al. observe in the context of trajectory analysis, standard grid-based embedding techniques often treat neighboring geographical cells as completely independent entities. When spatial topology is ignored, the model cannot leverage spatial proximity to determine path similarity, causing representations to fracture when data becomes sparse.

Similarly, Luo et al. point out that relying purely on dynamic interaction data (like human mobility flows) causes models to treat geographically adjacent but low-traffic zones as disconnected. To maintain geographic continuity, models must explicitly embed spatial topology alongside dynamic flows.

Injecting Physical Distance into the Loss Function

The most direct way to build a distance-preserving latent space is to rewrite the objective function to respect physical geometry. Instead of treating contrastive learning as a binary choice, researchers introduce continuous spatial metrics directly into the loss calculation.

Standard Contrastive Loss:
[Anchor] <---> [Positive (Same Class)]   vs.   [Anchor] <- - - - -> [Negative (Different Class)]
(Treats all negatives equally, ignoring physical distance)

Distance-Aware Contrastive Loss:
[Anchor] <---> [Close Negative] <- - - -> [Far Negative] <- - - - - - - - -> [Distant Negative]
(Penalizes predictions proportionally to their real-world physical or geodesic distance)

For example, in global visual geolocation, Gadi et al. modified the standard contrastive learning objective by incorporating geodesic distance (such as the Haversine distance) to weight the penalties for negative samples. By scaling the loss based on actual physical distance, the model is penalized less for guessing a nearby city and penalized severely for guessing a different continent. This simple geometric constraint forces the entire embedding space to align with real-world geographic topology.

A similar approach works for embodied AI agents navigating 3D environments. Jin et al. structured a latent space where the distance between vectors is explicitly proportional to the physical proximity of objects in the real world. By utilizing a continuous affinity value in their contrastive objective, the authors enabled models to develop an intrinsic sense of spatial relationships, allowing for precise object localization without requiring dense, pixel-wise computations.

This strategy of forcing the latent space to organize itself according to physical coordinates has been successfully applied across diverse modalities:

  • Medical Imaging: Bône et al. integrated spatial metadata (the depth of 2D slices) into contrastive objectives for cirrhosis classification, ensuring that latent distances reflect both semantic tissue similarity and geometric proximity.
  • Multimodal Language Models: Ding et al. designed contrastive objectives that align textual queries with structured map data, mapping geographically proximal locations to similar vector representations so the model can distinguish between textually similar but spatially distant entities.

Designing Geometry-Preserving Architectures

Beyond modifying loss functions, preserving spatial properties requires rethinking how coordinates and features are encoded into neural networks. In a systematic review of location encoding, Mai et al. formalize this as a foundational design principle: an effective spatial encoder must ensure that the similarity between vector embeddings decreases monotonically as the physical distance between coordinates increases.

In computer vision, maintaining this spatial layout often means resisting the urge to compress an image into a single, flat vector. Badrinath et al. demonstrate that generative image models preserve geometric structure and pose precisely because they maintain multi-dimensional latent encodings that correspond directly to the original image's layout, prioritizing spatial relationships over extreme vector compression.

When models must handle abstract, non-coordinate tokens (like discovered object parts), the challenge is bridging the gap between abstract embeddings and pixel-level coordinates. Xia et al. solved this by mapping sparse part-level tokens back to dense spatial feature maps to create spatial confidence maps. This alignment allows the model to enforce geometric constraints—such as spatial concentration and area normalization—ensuring that abstract tokens remain localized and distinct in physical space.

Why It Works: The Power of Inductive Bias

Distance-preserving spatial embedding works because it provides a powerful, mathematically rigorous inductive bias. Left to its own devices, a deep neural network will find statistical shortcuts in the training data that fail to generalize. By forcing latent distance to mirror physical distance, the model is constrained by the immutable laws of geometry.

Whether calculating the similarity of two trajectories, locating an object from a local view, or diagnosing a medical image, anchoring the latent space to physical coordinates ensures that the model's internal logic remains continuous, predictable, and aligned with the physical world.

Key Foundational Work

A Review of Location Encoding for GeoAI: Methods and Applications (2021) — contributed 2 concept elements to this direction.
Urban Region Profiling via Multi-Graph Representation Learning (2022) — contributed 2 concept elements to this direction.
History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 8 papers
  2. Version 1 · 2026-10-04 · 11 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_spatial_proximity_embedding_c01b8200,
  title = {spatial proximity embedding},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/spatial-proximity-embedding/rdr_c01b8200145449e9acc3c6a306045838},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 2, frozen text):

@misc{lacuna_spatial_proximity_embedding_c01b8200_v2,
  title = {spatial proximity embedding},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/spatial-proximity-embedding/rdr_c01b8200145449e9acc3c6a306045838/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake