Modeling Hierarchical Asymmetry in Semantic Spaces

95 concepts79 papers1 proposal

Standard representation learning models typically map words, entities, or concepts to single points in a Euclidean space. To determine how related two concepts are, these models compute geometric distance or cosine similarity. While this approach works well for symmetric relationships like synonymy (where huge and large share a mutual affinity), it fails fundamentally when encountering hierarchical structures.

Hierarchical relationships—such as hypernymy (is-a links), part-whole structures, and lexical entailment—are inherently directional and asymmetric. A robin is a bird, but a bird is not necessarily a robin. Because standard distance metrics are symmetric ($d(A, B) = d(B, A)$), point-based embeddings cannot naturally encode the one-way containment logic required for complex taxonomies. When forced into a symmetric space, models frequently confuse coordinate terms at the same level of a hierarchy (siblings like robin and penguin) with parent-child relationships, as noted by Haixun Wang et al. and Goran Glavaš et al.

To resolve this paradox, a major direction of research focuses on designing embedding spaces that natively support asymmetric relationship modeling.

Geometric Containment: From Points to Regions

The most direct way to build directionality into a space is to replace zero-dimensional points with multi-dimensional regions. In these frameworks, the asymmetric relationship "A implies B" is represented geometrically as "Region A is contained within Region B" ($A \subseteq B$).

Box Embeddings

Instead of vectors, concepts are modeled as hyperrectangles (boxes) in space. A box possesses both a position and a volumetric extent. As explored in A Single Vector Is Not Enough: Taxonomy Expansion via Box Embeddings (Extended Abstract) by Song Jiang et al. and the work on Insert or Attach: Taxonomy Completion via Box Embedding, box containment perfectly models the transitivity and asymmetry of taxonomies. If the box for robin is nested inside the box for bird, and bird is inside animal, the model mathematically guarantees that robin is an animal, without ever implying that every animal is a robin.

┌───────────────────────────────────────────┐
│ Animal                                    │
│    ┌─────────────────────────────────┐    │
│    │ Bird                            │    │
│    │    ┌───────────┐                │    │
│    │    │ Robin     │                │    │
│    │    └───────────┘                │    │
│    └─────────────────────────────────┘    │
└───────────────────────────────────────────┘

Intervals and Fuzzy Sets

This containment logic can take other geometric forms. Jean-Philippe Bernardy et al. demonstrate that mathematical intervals can represent deep hierarchies like WordNet with near-perfect accuracy. To accommodate the graded, non-binary nature of human language—where a robin is perceived as a more "typical" bird than a penguin—Fred Xu introduces FUSE: Measure-Theoretic Compact Fuzzy Set Representation for Taxonomy Expansion. By modeling concepts as fuzzy sets, the framework allows for soft, overlapping boundaries while strictly maintaining measure-theoretic volume constraints to govern directional inclusion.

Probabilistic Envelopes and Order Embeddings

An alternative to strict geometric boundaries is the use of continuous probability distributions or coordinate-wise constraints to enforce directional hierarchies.

Density Inclusion

Instead of mapping data to fixed regions, concepts can be modeled as probability distributions. In Generalizing Point Embeddings using the Wasserstein Space of Elliptical Distributions, Boris Muzellec et al. represent complex objects as elliptical distributions, using the 2-Wasserstein metric to capture uncertainty and hierarchical depth. Broad, high-variance distributions represent general categories, while narrow, low-variance distributions represent specific instances. Similarly, Zhibin Duan et al. implement TopicNet: Semantic Graph-Guided Topic Discovery using Gaussian distributions and thresholded divergence measures, forcing broader parent distributions to envelop their more specific descendants.

Order Embeddings

Directionality can also be baked directly into the coordinates of a vector space by establishing a partial order. In an order-embedding space, a specific concept $x$ and a general category $y$ are constrained by a coordinate-wise inequality condition ($x_i \ge y_i$ for all dimensions $i$). This creates a directed cone of entailment.

Hang Zhao et al. apply this in Open Vocabulary Scene Parsing, and Jônatas Wehrmann utilizes it in Order embeddings and character-level convolutions for multimodal alignment. This partial-order geometry ensures that a highly detailed caption (e.g., "a small brown robin sitting on a wooden fence") is recognized as a specific subset of a broader visual category ("a bird outdoors").

Breaking Symmetry with Operators and Loss Functions

When models must operate within standard vector spaces, researchers break the symmetry of the dot product by introducing specialized operators, dual-role representations, or asymmetric loss formulations.

Bilinear Scoring Functions

The standard dot product $x \cdot y$ is commutative ($x^T y = y^T x$). By inserting a transformation matrix $W$, we get a bilinear form $x^T W y$. If $W$ is asymmetric, then $x^T W y \neq y^T W x$. Shikhar Murty leverages this property in Finer Grained Entity Typing with TypeNet, demonstrating that bilinear scoring functions successfully capture the directional logic where a specific subtype implies a general parent type, but the reverse is invalid.

Dual-Vector Representations

Another strategy is to decouple an entity's representation based on its position in the relation. Haixun Wang et al. assign two distinct vectors to every word: one representing its role as a general category (a hypernym) and one as a specific instance (a hyponym). The training objective minimizes the distance from a specific term's hyponym vector to a general term's hypernym vector, explicitly encoding directionality into the optimization landscape.

Asymmetric Losses and Architectures

Symmetry can also be broken at the loss level or via architectural gating. Soumen Chakrabarti et al. introduce an asymmetric loss term during relation alignment in Joint Completion and Alignment of Multilingual Knowledge Graphs to prevent models from assuming perfect synonymy when one relation is a subset of another. For lexical entailment, Marek Rei et al. propose the Supervised Directional Similarity Network (Scoring Lexical Entailment with a Supervised Directional Similarity Network), which uses an explicit gating mechanism to transform general-purpose word embeddings into a specialized directional space.

The Distributional Inclusion Hypothesis

The empirical foundation for extracting these asymmetric relationships from text relies on the Distributional Inclusion Hypothesis, originally formalized by Maayan Geffet et al. in The Distributional Inclusion Hypotheses and Lexical Entailment.

While standard distributional semantics assumes that words appearing in similar contexts share similar meanings, the inclusion hypothesis states that if a specific term $A$ entails a general term $B$, then the characteristic linguistic contexts of $A$ should be a subset of the contexts of $B$.

Contexts of "Doctor"  : [ "works in hospital", "treats patients", "prescribes medicine" ]
                                     ▲
                                     │ (Inclusion)
                                     │
Contexts of "Surgeon" : [ "works in hospital", "treats patients", "performs surgery" ]

To operationalize this, Lili Kotlerman et al. design an asymmetric similarity measure based on Average Precision (Directional Distributional Similarity for Lexical Expansion) to quantify how well a specific word's context features are encompassed by a broader term. Similarly, Eiko Yamamoto et al. utilize a frequency-informed "gray-scale" inclusion approach in Extraction of Hierarchies Based on Inclusion of Co-occurring Words with Frequency Information to determine the exact direction of parent-child links in automatically generated ontologies. By shifting the mathematical focus from overlap (similarity) to containment (inclusion), these models successfully extract the directed, hierarchical backbone of human language.

History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_asymmetric_relationship_modeling_hierarc_0f19d904,
  title = {asymmetric relationship modeling (hierarchical directionality)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/asymmetric-relationship-modeling-hierarchical-directionality/rdr_0f19d904a5174687b8d09e722fa74336},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_asymmetric_relationship_modeling_hierarc_0f19d904_v1,
  title = {asymmetric relationship modeling (hierarchical directionality)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/asymmetric-relationship-modeling-hierarchical-directionality/rdr_0f19d904a5174687b8d09e722fa74336/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake