Tensor-Based Compositional Distributional Semantics

This direction is merged into another direction: tensor-based compositional semantics. Its last version stays readable and citable here.

Version 1 of 118 papers30 conceptsfrozen 2026-10-04

Standard language models often treat sentences as "bags of words" or sequences where meaning is derived through proximity. However, human language is inherently structural: verbs act as functions that transform the meaning of their subjects and objects. To capture this, researchers have developed a framework that merges the statistical power of word vectors with the structural rigor of formal logic using high-order tensors.

The Functional Nature of Verbs

In traditional distributional semantics, every word is a vector. While this works for nouns like "cat" or "dog," it fails to capture the role of relational words. Dimitri Kartsaklis and Mehrnoosh Sadrzadeh argue in Prior Disambiguation of Word Tensors for Constructing Sentence Vectors that verbs should be viewed as functions.

Instead of a vector, a transitive verb is represented as a third-order tensor. This tensor acts as a multi-linear map: when you "plug in" the vectors for a subject and an object, the tensor contracts them to produce a new vector representing the sentence. This ensures that "The dog chased the cat" and "The cat chased the dog" result in different mathematical representations, even though they contain the exact same words.

Bridging Logic and Vectors

The challenge in NLP has long been choosing between the "fuzzy" similarity of vectors and the "hard" rules of logic. Yarin Gal describes this direction as Compositional Distributional Semantics. By using tensor algebra, models can simulate logical operations like negation and quantification within a continuous vector space.

Christopher Potts and Samuel R. Bowman demonstrated in Recursive Neural Networks for Learning Logical Semantics that these multiplicative interactions are essential. Simple addition (adding "not" to "happy") cannot capture the inversion of meaning; you need the multiplicative power of tensors to "flip" the semantic orientation of a vector.

Learning Meaning through Contraction

How do we actually find the numbers for these large verb tensors? Tamara Polajnar and Stephen Clark show in Learning Type-Driven Tensor-Based Meaning Representations that we can learn these tensors using standard optimization. By treating the sentence-level output as a "plausibility score," the model learns to adjust the internal values of the verb tensor so that meaningful subject-verb-object combinations (like "waiter serves soup") are mathematically distinct from nonsensical ones ("soup serves waiter").

Beyond Simple Sentences

This structural approach is particularly powerful for complex linguistic phenomena like ellipsis—where parts of a sentence are omitted. In Evaluating Composition Models for Verb Phrase Elliptical Sentence Embeddings, Gijs Wijnholds demonstrates that tensor models can "copy" the missing semantic information from a previous clause into the empty slot of an elliptical one. Because the model understands the functional role of the verb, it can reconstruct the implied meaning more accurately than a "black-box" neural encoder.

By treating grammar as a blueprint for mathematical operations, this research direction ensures that the composition of meaning is not just a sum of its parts, but a structured transformation.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_tensor_based_compositional_distributiona_8f73b6fa,
  title = {Tensor-Based Compositional Distributional Semantics},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/tensor-based-compositional-distributional-semantics/rdr_8f73b6fa558148b794355f743a97c409},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_tensor_based_compositional_distributiona_8f73b6fa_v1,
  title = {Tensor-Based Compositional Distributional Semantics},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/tensor-based-compositional-distributional-semantics/rdr_8f73b6fa558148b794355f743a97c409/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}