Knowledge graphs and relational networks store vast amounts of real-world information as discrete, symbolic structures—typically formatted as triples like (Subject, Predicate, Object). While these representations are highly intuitive for human curation and symbolic logic, they present a fundamental challenge for modern machine learning. As noted by Dutta et al., discrete graphs lack the algebraic properties of vector spaces. You cannot easily compute gradients, determine continuous distances, or pass a symbolic graph directly into a deep neural network without encountering a combinatorial explosion of features (Cvetkov-Iliev et al.; Han et al.).
Graph representation learning and knowledge graph embedding (KGE) solve this by mapping discrete entities and relations into continuous, low-dimensional vector spaces (Song; Peng et al.). By translating symbolic data into dense numerical vectors, this approach bridges the gap between structured human knowledge and differentiable neural architectures.
The Core Intuition: Geometry as Logic
The central idea behind knowledge graph embeddings is to replace rigid symbolic reasoning with geometric operations. Instead of traversing a graph using discrete logical rules, we use geometric properties in the embedding space to perform reasoning and link prediction (Song).
In this continuous coordinate system, semantic relevance is represented by geometric distance or directional alignment. For example, in a biomedical knowledge graph containing drugs, targets, and pathways, the vectors for related entities are optimized to sit close to one another. As Malusare et al. explain, this spatial arrangement allows complex relational data to function as a structured coordinate system where geometric proximity directly mirrors biological relevance.
To achieve this, embedding models optimize a score function for every triple. Depending on the architecture, these functions can be:
- Translation-based: Treating relations as translations in the vector space, where the vector of the subject plus the vector of the relation should approximate the vector of the object ($\vec{s} + \vec{r} \approx \vec{o}$).
- Bilinear/Matrix Factorization: Using tensor decomposition methods like DistMult to capture multiplicative interactions between entity and relation vectors across billions of triples (Färber et al.).
Preserving Topology and Latent Neighborhoods
A successful embedding must compress high-dimensional graph structures into fixed-size vectors without losing the underlying structural context (Myklebust et al.; Pei et al.). This means the embedding of a single node must encode not just its identity, but its "latent neighborhood"—the web of connections surrounding it (Mondal).
[Discrete Graph Structure] [Continuous Vector Space]
(Drug A) --[interacts]--> (Target B) =======> Vec(Drug A) + Vec(interacts) ≈ Vec(Target B)
| (Spatial proximity encodes latent pathways)
[path]
v
(Disease C)
By capturing these latent neighborhoods, embeddings allow models to generalize beyond explicit paths. Traditional graph algorithms often rely on shared neighbors or explicit path-walking to find connections. However, Crichton et al. demonstrate that embedding-based approaches significantly outperform simple path heuristics when predicting "unexpected" links—connections between entities that share no immediate common neighbors but occupy structurally similar positions in the global network topology.
Why the Continuous Shift Works
Transitioning from discrete graph mining to continuous network embeddings represents a major paradigm shift in scalability and flexibility (Pei et al.).
- Avoiding Combinatorial Explosion: Traditional automated feature engineering on relational databases requires combinatorially exploring paths and links. Embeddings provide a compact, fixed-size representation that distinguishes between different types of links between the same entities, bypassing this computational bottleneck (Cvetkov-Iliev et al.).
- Downstream Compatibility: Once a graph is mapped to a low-dimensional vector space, the resulting embeddings serve as standard continuous features. They can be fed directly into downstream neural networks, classifiers, or generative workflows (Celebi et al.; Peng et al.).
- Statistical Generalization: Early work by Sebag demonstrated that mapping complex relational structures into vector spaces allows simple statistical learners to achieve classification accuracies comparable to specialized inductive logic programming systems. The continuous space smooths out the discrete, brittle nature of symbolic logic, allowing models to tolerate noise and missing data.
Ultimately, knowledge graph embeddings work because they transform the problem of discrete structural search into a problem of continuous geometric optimization, unlocking the full power of modern deep learning for relational data.
Key Foundational Work
DistanceJoin: Pattern Match Query In a Large Graph Database (2009) — contributed 1 concept elements to this direction.