Learning Representations in Hypercomplex Quaternion Spaces

27 concepts20 papers2 proposals

In standard machine learning architectures, entities—whether they are nodes in a graph, users in a recommendation system, or concepts in a knowledge base—are typically represented as vectors in a real-valued Euclidean space $\mathbb{R}^d$. In this conventional setup, each dimension of the vector is treated as an independent scalar. To capture interactions between different latent features, models must rely on dense weight matrices (such as fully connected layers) to explicitly mix the dimensions.

This approach introduces two significant challenges: it causes a rapid explosion in parameter count, and it struggles to naturally encode geometric transformations like rotations. Geometric rotations are crucial for modeling relational properties such as symmetry, inversion, and hierarchical depth.

To overcome these limitations, a powerful paradigm shift has emerged: mapping representations into hypercomplex spaces, specifically Quaternion space ($\mathbb{H}$). By leveraging the unique algebraic properties of quaternions, models can capture intricate, multi-dimensional feature interactions and structural dependencies natively within the representation space itself—often with a fraction of the parameters required by Euclidean models.

The Intuition Behind Quaternion Space

To understand why quaternions are so effective, it helps to look at how number systems expand. A real number represents a point on a one-dimensional line. A complex number ($a + bi$) introduces a second dimension via an imaginary unit ($i^2 = -1$), creating a two-dimensional plane where multiplication naturally performs a geometric rotation.

A quaternion extends this concept even further into a four-dimensional hypercomplex space. It consists of one real component and three imaginary components:

$$q = a + bi + cj + dk$$

where $i$, $j$, and $k$ are distinct imaginary units that satisfy the fundamental formula discovered by William Rowan Hamilton:

$$i^2 = j^2 = k^2 = ijk = -1$$

Crucially, quaternion multiplication—known as the Hamilton product—is non-commutative (for example, $ij = k$, but $ji = -k$). This means the order of operations matters, providing an elegant mathematical framework for modeling directional, ordered, or causal relationships.

The core reason why quaternions are highly expressive is how the Hamilton product forces components to interact. When you multiply two quaternions, their components do not multiply independently. Instead, they undergo a highly structured cross-dimensional coupling where every component of the first quaternion interacts with every component of the second. As noted by Zheyu Chen et al. in Hypercomplex Prompt-aware Multimodal Recommendation, this multi-component structure allows a single quaternion embedding to capture diverse, multi-faceted feature subspaces in parallel, rather than relying on flat, uncoupled real-valued vectors.

Modeling Complex Relations as 3D Rotations

The geometric properties of quaternions make them exceptionally well-suited for Knowledge Graph Embeddings (KGE). In a knowledge graph, facts are represented as triplets: $(\text{head}, \text{relation}, \text{tail})$. Traditional models often treat relations as simple translations ($h + r \approx t$).

By transitioning to a hypercomplex space, Shuai Zhang et al. demonstrated in Quaternion Knowledge Graph Embeddings. that relations can be modeled as 3D rotations within the 4D quaternion space via the Hamilton product. Because quaternions can smoothly and compactly represent rotations around any arbitrary axis in 3D space, a relation-defined rotation can simultaneously capture intricate logical patterns like symmetry, antisymmetry, and inversion.

Euclidean Space:  [Head Vector] ----+ Relation Translation ----> [Tail Vector]

Quaternion Space: [Head Vector] ----↺ Relation 3D Rotation ----> [Tail Vector]

This geometric framework has been extended to capture even more nuanced structural properties:

Preventing Topology Collapse and Over-Smoothing in Graphs

Beyond knowledge graphs, hypercomplex representations offer significant advantages for Graph Neural Networks (GNNs) and collaborative filtering. In standard GNNs, aggregating neighbor features using real-valued operations frequently leads to two common failure modes: "over-smoothing" (where node representations become indistinguishable after a few layers) and "topology collapse" in directed graphs (where the directionality of edges is lost).

Quaternion algebra naturally mitigates these issues. Stefano Fiorini et al. introduced a novel framework in Graph Learning in 4D: a Quaternion-valued Laplacian to Enhance Spectral GCNs, defining a Quaternionic Laplacian for spectral GCNs. The four-dimensional interaction preserves the directional flow and intensity of information along reciprocal edges, preventing structural information from washing out.

Similarly, Junyang Chen et al. showed in Attributed Graph Clustering via Generalized Quaternion Representation Learning that modeling node attributes as rotations in a hypercomplex space increases the degrees of freedom available for feature coupling. This allows the network to decouple distinct data dimensions, mitigating over-smoothing without requiring excessive layer depth.

In recommendation systems, where user-item interactions form complex bipartite graphs, models like those proposed by Lina Yao et al. in Quaternion Collaborative Filtering for Recommendation and Yaxing Fang et al. in Quaternion-based graph convolution network for recommendation replace standard Euclidean vectors with quaternions. By leveraging the Hamilton product, these architectures capture intricate, non-linear user preferences and structural features of interaction graphs that are typically distorted or lost in flat real-valued spaces.

Why It Works: Extreme Parameter Efficiency

The mathematical overhead of hypercomplex algebra yields a massive practical dividend: dramatic parameter efficiency.

In a standard real-valued vector space, if you want to allow full interaction between two 4-dimensional representations, you must multiply them by a $4 \times 4$ dense matrix, requiring 16 learnable parameters.

In contrast, when you perform a Hamilton product between two quaternions, the cross-dimensional mixing is governed entirely by the algebraic rules of $i, j,$ and $k$. The operation can be written as a block matrix multiplication where the same 4 parameters ($a, b, c, d$) are structurally reused across the matrix:

$$\begin{pmatrix} \hfill a & -b & -c & -d \\ \hfill b & \hfill a & -d & \hfill c \\ \hfill c & \hfill d & \hfill a & -b \\ \hfill d & -c & \hfill b & \hfill a \end{pmatrix}$$

Because of this built-in weight sharing, a quaternion layer achieves the same level of cross-dimensional expressiveness as a dense real-valued layer but requires only 4 learnable parameters instead of 16—a 75% reduction.

This structural efficiency is consistently borne out in empirical results. For example, the recommendation framework developed by Thanh Tran et al. in Quaternion-Based Self-Attentive Long Short-Term User Preference Encoding for Recommendation achieved superior robustness and accuracy while using 75% fewer parameters than standard Euclidean baselines. Similarly, the temporal knowledge graph model proposed by Jiujiang Guo et al. required up to 10x fewer parameters than competing architectures.

By embedding the geometric and structural rules of a domain directly into the algebraic properties of the representation space, hypercomplex learning bypasses the need for massive, fully connected parameter matrices, proving that more expressive AI models do not always require more parameters.

Key Foundational Work

Quaternion Knowledge Graph Embeddings. (2019) — contributed 1 concept elements to this direction.
Quaternion-Based Self-Attentive Long Short-Term User Preference Encoding for Recommendation (2020) — contributed 1 concept elements to this direction.
Quaternion Collaborative Filtering for Recommendation (2019) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_quaternion_embeddings_532d897e,
  title = {quaternion embeddings},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/quaternion-embeddings/rdr_532d897e44b049b59546a3e2d2229c79},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_quaternion_embeddings_532d897e_v1,
  title = {quaternion embeddings},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/quaternion-embeddings/rdr_532d897e44b049b59546a3e2d2229c79/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake