Unsupervised Discovery of Latent Hierarchical Structures

374 concepts325 papers2 proposals

Real-world data is rarely "flat." Human knowledge, language, and physical objects naturally organize into multi-level taxonomies where broad concepts encompass specific sub-types. For instance, as noted by the authors of CNN-RNN: a large-scale hierarchical image classification framework, a golden retriever is a dog, which is a mammal, which is an animal.

Traditional representation learning techniques often ignore this structure, mapping entities into flat vector spaces or relying on rigid, manually curated taxonomies. When these explicit labels are missing, models fail to capture the multi-resolution nature of the data. To address this, Dietrich Klakow et al. demonstrated in Parameter Free Hierarchical Graph-Based Clustering for Analyzing Continuous Word Embeddings that continuous spaces contain an inherent semantic "geography" that can be mapped into navigable graphs without manual intervention. The goal of latent hierarchy discovery is to autonomously expose these multi-level relationships directly from raw, unstructured data.

Geometric Inductive Biases for Depth and Granularity

To discover hierarchies without labels, we must first use mathematical spaces that naturally accommodate tree-like structures. Standard Euclidean space scales poorly for this task: the number of nodes in a tree grows exponentially with depth, whereas the volume of a Euclidean space grows only polynomially. This mismatch forces models to compress hierarchical relationships, losing structural fidelity.

To solve this volume mismatch, researchers turn to non-Euclidean geometries. In Learning Continuous Hierarchies in the Lorentz Model of Hyperbolic Geometry, Maximilian Nickel et al. leverage the Lorentz model of hyperbolic space. Because hyperbolic space expands exponentially, it provides the exact geometric capacity needed to embed tree structures naturally using simple similarity scores. In this space, an entity's distance from the origin correlates directly with its hierarchical abstraction. This geometric intuition is also utilized by Lingxiao Li et al. in The Euclidean Space is Evil: Hyperbolic Attribute Editing for Few-shot Image Generation, where mapping latent codes to a Poincaré disk allows the radius to control semantic abstraction and diversity without supervised labels.

Euclidean Space (Polynomial Growth)       Hyperbolic Space (Exponential Growth)
       o        o        o                       \       |       /
        \       |       /                         o      o      o
         o------o------o                           \    / \    /
        /       |       \                           \  /   \  /
       o        o        o                            o     o
(Crowding occurs at deeper levels)         (Natural expansion matches tree growth)

Alternative geometric representations use coordinates or volumes to explicitly separate depth from identity. For instance, Zhanqiu Zhang et al. introduce Learning Hierarchy-Aware Knowledge Graph Embeddings for Link Prediction, which maps entities into a polar coordinate system. Here, the radial coordinate explicitly models hierarchical depth, while the angular coordinate distinguishes between sibling entities at the same level. To capture both containment and specific meaning, Zijie Huang et al. propose Concept2Box: Joint Geometric Embeddings for Learning Two-View Knowledge Graphs, representing high-level concepts as geometric boxes to capture granularity and inclusion, and fine-grained entities as vectors.

Generative and Probabilistic Branching

Instead of just mapping data into a static geometry, we can model the data-generation process itself as a branching or evolutionary tree. This approach assumes that observed variations are the result of latent mutations from canonical source concepts.

In Name Phylogeny: A Generative Model of String Variation, Nicholas Andrews et al. treat an unorganized collection of strings as an evolutionary tree (a phylogeny). By modeling how canonical entities mutate into derivatives without explicit parent-child pairs, the system learns latent hierarchical string variations (e.g., tracing how "William" branches into "Bill" or "Billy").

For continuous data, probabilistic mixture models can be integrated directly into deep architectures. In Hierarchically Clustered Representation Learning, Kyungwoo Song et al. combine a Hierarchical Gaussian Mixture Model (HGMM) with a Variational Autoencoder (VAE). This allows the network to jointly optimize feature learning and hierarchical clustering, forcing the latent space to conform to a tree-like organization. This generative approach is echoed in the classic work Semi-Supervised Learning with Trees, where the authors model label propagation as a stochastic mutation process over an inferred latent tree backbone, showing that discovering this tree structure provides a powerful inductive bias for semi-supervised classification.

Joint Optimization of Structure and Embeddings

A major paradigm shift in this field is moving away from two-stage pipelines—where clustering is performed after embeddings are learned—toward frameworks that jointly discover the hierarchy while learning the representations. This prevents the embedding space from discarding structural nuances early in training.

In eTREE: Learning Tree-structured Embeddings, Faisal M. Almutairi et al. treat the unknown tree structure as a "soft" regularizer within a matrix factorization framework. This allows the model to handle highly sparse data by co-opting the latent hierarchy to guide the embedding process. Similarly, Carl Yang et al. develop TAXOGAN: Hierarchical Network Representation Learning via Taxonomy Guided Generative Adversarial Networks (Extended Abstract) to jointly discover latent hierarchical structures and node embeddings using a generative adversarial framework when a formal taxonomy is completely missing.

Even when only partial or coarse supervision is available, models can infer the missing structural layers. In Fine-grained Classes and How to Find Them, Matej Grcic introduces a framework that discovers hidden fine-grained classes using only coarse labels. By alternating between training a standard classifier and solving a discrete optimization problem, the model maps fine-grained sub-types to their coarse parents, demonstrating that broad supervision is sufficient to bootstrap latent hierarchy discovery.

Inducing Recursive and Part-Whole Structures

Hierarchy also manifests as recursive depth or part-whole decompositions in sequential and spatial data. In these domains, the challenge is to discover how smaller components combine to form larger structures without explicit syntactic or structural labels.

In natural language processing, Shikhar Murty et al. present Pushdown Layers: Encoding Recursive Structure in Transformer Language Models, which replaces standard self-attention with an explicit, stack-based memory mechanism. This architecture tracks the recursive depth of tokens in a "stack tape," enabling the model to autonomously discover latent recursive patterns and syntactic hierarchies without pre-computed syntactic labels.

Input Tokens:      [ The   cat   that   chased   the   rat   ran ]
Stack Depth:         |      |      |       |      |     |     |
                     |      |      +-------+------+-----+     |
                     |      +---------------------------------+
                     +----------------------------------------+

In computer vision and 3D shape analysis, the hierarchy represents how objects are composed of parts. In RIM-Net: Recursive Implicit Fields for Unsupervised Learning of Hierarchical Shape Structures, Chengjie Niu et al. combine recursive implicit fields with a part decoder to decompose 3D shapes into a multi-level hierarchy of parts (e.g., decomposing a chair into a back and seat, and then the seat into legs) entirely without human-labeled data. By enforcing a recursive decomposition budget, the network learns to isolate structural boundaries naturally.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 325 papers
  2. Version 1 · 2026-10-04 · 418 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_automated_hierarchy_discovery_latent_hie_7fe58c48,
  title = {automated hierarchy discovery (latent hierarchy discovery)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/automated-hierarchy-discovery-latent-hierarchy-discovery/rdr_7fe58c48f6f84477b7df9183534f8ddd},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 2, frozen text):

@misc{lacuna_automated_hierarchy_discovery_latent_hie_7fe58c48_v2,
  title = {automated hierarchy discovery (latent hierarchy discovery)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/automated-hierarchy-discovery-latent-hierarchy-discovery/rdr_7fe58c48f6f84477b7df9183534f8ddd/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake