Complex Valued Word Embeddings

45 concepts24 papers5 proposals

Traditional natural language processing relies on real-valued vector spaces to represent the meanings of words. While these representations excel at capturing surface-level semantic similarity, they struggle with compositionality—the principle that the meaning of a complex expression is determined by the meanings of its parts and the rules used to combine them. In standard vector spaces, word combination is typically modeled via linear operations like vector addition or multiplication. However, human language is full of non-linear interactions: phrases like "ivory tower" develop emergent meanings, sarcastic remarks invert literal definitions, and polysemous words shift drastically depending on context.

To capture these nuances, a growing body of research introduces complex-valued word embeddings. By mapping language into a complex Hilbert space, these models leverage the mathematical machinery of quantum mechanics—specifically amplitude, phase, superposition, and interference—to represent the fluid, contextual, and interactive nature of human language.

Decoupling Meaning via Amplitude and Phase

The core intuition behind complex-valued embeddings lies in the geometry of the complex plane. A complex number $z$ can be expressed in polar form as $z = r e^{i\theta}$, where $r$ is the magnitude (amplitude) and $\theta$ is the phase angle. This architecture provides two independent channels of information per dimension, allowing models to decouple distinct linguistic facets that real-valued vectors inherently conflate.

        Imaginary Axis
              ^
              |      . z = r * e^(i*theta)
              |     / |
              |    /  |
              |   /   |  Nuance / Sentiment / Context
              |  /    |  (Encoded in Phase angle 'theta')
              | /     |
              +-------+------------> Real Axis
                 Core Lexical Meaning
                 (Encoded in Amplitude 'r')

As pioneered by Qiuchi Li et al. in Quantum-Inspired Complex Word Embedding, the amplitude can represent the primary semantic strength or lexical importance of a concept, while the phase captures auxiliary features or contextual orientation. This separation is particularly powerful for sentiment and emotion analysis. For instance, Guangcheng Liu et al. in A Quantum-inspired Complex-valued Representation for Encoding Sentiment Information (Student Abstract) and Jinjing Shi et al. in Two End-to-End Quantum-Inspired Deep Neural Networks for Text Classification demonstrate that encoding a word's literal definition in the amplitude and its emotional valence in the phase allows a network to cleanly distinguish between words that share similar contexts but possess opposite emotional polarities.

This decoupling strategy extends beyond sentiment. In sequential recommendation systems, Zhen Tian et al. developed EulerFormer: Sequential User Behavior Modeling with Complex Vector Attention, which uses the phase angle to preserve an item's core identity while using the magnitude to scale its importance or confidence within a user's history.

Alternatively, when viewed in Cartesian coordinates ($a + bi$), the real and imaginary parts can serve as distinct representational tiers. Jing Li et al. demonstrate in AoE: Angle-optimized Embeddings for Semantic Textual Similarity that the real component can capture surface-level lexical appearance, while the imaginary component isolates subtle semantic nuances. This prevents distinct semantic features from collapsing into one another during the embedding process.

Resolving Ambiguity with Quantum Superposition

In standard NLP, a polysemous word like "bank" (a financial institution or the side of a river) is often forced into a single static vector that averages its disparate meanings. Complex-valued Hilbert spaces solve this by treating words as quantum states that exist in a superposition of multiple semantic basis vectors.

According to Peng Zhang et al. in A Quantum Many-body Wave Function Inspired Language Modeling Approach, representing words as state vectors allows ambiguous terms to be modeled as a probabilistic combination of distinct semantic states. Similarly, Yiwei Chen et al. utilize this property in Quantum Language Model With Entanglement Embedding for Question Answering to model a superposition of multiple potential meanings or "sememes."

When an ambiguous word is isolated, it retains its full spectrum of potential meanings. However, when it interacts with surrounding context, the superposition collapses into a specific meaning. In QSAN: A Quantum-probability based Signed Attention Network for Explainable False Information Detection, the authors leverage this mechanism to capture how word meanings dynamically shift or resolve in the presence of contradictory contexts, such as user comments debunking a fake news story.

Semantic Composition as Wave Interference

The most compelling justification for complex-valued embeddings is their ability to model semantic interference. When two waves meet, they can reinforce each other (constructive interference) or cancel each other out (destructive interference) depending on their relative phases. Complex word embeddings bring this physical principle to language.

Constructive Interference (Reinforcement)
Wave 1 (Phase 0°)   : /\/\/\
Wave 2 (Phase 0°)   : /\/\/\
Result (Amplified)  : /  \ /  \ /  \

Destructive Interference (Cancellation/Inversion)
Wave 1 (Phase 0°)   : /\/\/\
Wave 2 (Phase 180°) : \/\/\/
Result (Canceled)   : ----------------

As noted by William Blacoe et al. in A Quantum-Theoretic Approach to Distributional Semantics, phase interactions allow specific syntactic contexts to either reinforce or suppress particular senses of a word. This provides an elegant solution for non-linear linguistic phenomena like sarcasm and negation:

Breaking the Symmetry of Text Matching

Beyond compositionality, complex spaces offer distinct geometric advantages over real-valued spaces for downstream tasks like text matching and similarity optimization.

Overcoming Cosine Saturation

Traditional sentence embedding models optimize using cosine similarity. However, cosine functions suffer from a "saturation zone" where gradients vanish, making it difficult for a model to distinguish between texts that have high lexical overlap but subtle semantic differences. To bypass this, Jing Li et al. introduced AnglE-optimized Text Embeddings. By mapping embeddings into a complex space and optimizing the angle between vectors rather than their cosine, the model maintains strong gradients and preserves fine-grained semantic distinctions.

Modeling Asymmetrical Relationships

Real-valued dot products are inherently symmetric ($A \cdot B = B \cdot A$). This symmetry is a liability for directional tasks like question-answering (a question matches an answer, but an answer does not "match" a question in the same way) or textual entailment.

Yi Tay et al. address this in Hermitian Co-Attention Networks for Text Matching in Asymmetrical Domains by replacing the standard dot product with a Hermitian inner product (which involves taking the complex conjugate of the second vector). Because the Hermitian inner product is conjugate-symmetric ($<A, B> = <B, A>^*$), it naturally encodes directionality. The imaginary component functions as a latent representational space for context or memory, allowing the network to natively handle asymmetrical text matching without inflating the model's parameter count.

Key Foundational Work

Quantum-Inspired Complex Word Embedding (2018) — contributed 5 concept elements to this direction.
A Quantum-Theoretic Approach to Distributional Semantics (2013) — contributed 1 concept elements to this direction.
A Quantum Many-body Wave Function Inspired Language Modeling Approach (2018) — contributed 1 concept elements to this direction.
History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 24 papers
  2. Version 1 · 2026-10-04 · 37 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_complex_valued_word_embeddings_27815343,
  title = {complex-valued word embeddings},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/complex-valued-word-embeddings/rdr_2781534353614a689812f5ab3a8ee7e7},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 2, frozen text):

@misc{lacuna_complex_valued_word_embeddings_27815343_v2,
  title = {complex-valued word embeddings},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/complex-valued-word-embeddings/rdr_2781534353614a689812f5ab3a8ee7e7/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-22},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake