Schema Agnostic Relation Discovery from Unstructured Text

57 concepts48 papers2 proposals

Imagine trying to build a comprehensive database of every fact on the internet. The traditional approach to Information Extraction (IE) requires a human-defined schema: you specify a fixed set of target relations—such as is_headquartered_in(Organization, Location) or acquired(Company, Company)—and manually label thousands of sentences to train a classifier.

This supervised paradigm breaks down when scaled to the web or specialized domains like scientific literature or clinical trials. As noted by Kata Gábor et al. in Détection et classification non supervisées de relations sémantiques dans des articles scientifiques (Unsupervised Classification of Semantic Relations in Scientific Papers), creating human-annotated gold standards for every niche domain is prohibitively expensive and fundamentally unscalable.

The solution is Schema-Agnostic Relation Discovery (often called Open Information Extraction, or OIE). Instead of forcing natural language into a predefined box, these systems allow the schema to emerge organically from the text itself. By extracting arbitrary relationships directly from linguistic structures, machines can read at a global scale without human intervention.


Extracting Structure from Syntax

How do you extract a relationship if you don't know what you are looking for? The core insight is that while the meaning of relationships varies infinitely, the structural patterns used to express them in natural language are highly regular.

In their seminal work on TextRunner: Open Information Extraction on the Web, Alexander Yates et al. bypassed deep semantic parsing in favor of a single-pass architecture that extracts relational triples in the form of (argument 1, relation string, argument 2). For example, given the sentence "Apples contain vitamin C," the system identifies "contain" as the predicate based purely on its syntactic position between two noun phrases.

[Apples] ---> (contain) ---> [vitamin C]
  Arg1          Relation         Arg2

This mechanism effectively treats every unique verb or context string as an emergent database predicate. As Mike Cafarella et al. detailed in Structured Querying of Web Text Data: A Technical Challenge, this transforms unstructured text into a queryable probabilistic database. Instead of navigating simple document links, users can execute SQL-like queries over extracted tables of facts.

This structural approach can be surprisingly precise even with minimal abstraction. Early work by Uri Zernik in Shipping Departments vs. Shipping Pacemakers: Using Thematic Analysis to Improve Tagging Accuracy demonstrated that by analyzing how much word pairs vary across massive unannotated corpora, systems can automatically distinguish active thematic relationships (like "shipping pacemakers") from fixed noun collocations (like "shipping departments") without manual labeling.


Organizing the Open Schema: Clustering and Representation

While treating every raw text string as a relation scales effortlessly, it introduces a massive redundancy problem. The phrases "was developed by", "was created by", and "is the brainchild of" all mean essentially the same thing. To make an open schema useful, the system must group these synonymous expressions together.

One way to achieve this is by learning distributional relation vectors. Rather than relying on rigid string matching, Shoaib Jameel et al. showed in Unsupervised Learning of Distributional Relation Vectors that high-quality relational features can be captured by vectorizing the context words that sit between entity pairs.

Once represented in a vector space, unsupervised clustering algorithms can group similar relations together:

Overcoming Cluster Collapse

When training deep neural networks to perform this clustering discriminatively, models face a major mathematical trap: representation collapse. Left unconstrained, a deep model will quickly discover a trivial solution—either assigning every sentence to the exact same generic relation category (maximizing confidence but destroying diversity) or scattering them randomly.

To stabilize this process, Étienne Simon et al. introduced RelDist in Unsupervised Information Extraction: Regularizing Discriminative Approaches with Relation Distribution Losses. RelDist applies two competing regularization losses during training:

  1. Skewness Loss: Forces the model to be highly confident in its individual predictions, pushing the probability distribution of a given instance toward a single relation type.
  2. Dispersion Loss: Forces the average prediction across the entire dataset to be uniform, ensuring that the model utilizes all available relation categories rather than collapsing into one.

$$\mathcal{L}_{\text{RelDist}} = \mathcal{L}_{\text{skewness}} (\text{be confident}) + \mathcal{L}_{\text{dispersion}} (\text{be diverse})$$


The Surprising Power of Entity Types and Language Models

As deep learning models grew more complex, researchers began to question whether heavy neural architectures were strictly necessary for unsupervised relation discovery.

In Revisiting Unsupervised Relation Extraction, Thy Thy Tran et al. revealed a striking finding: simply grouping text spans based on the types of the entities involved (e.g., knowing that a relation connects a PERSON and a LOCATION) can outperform state-of-the-art deep clustering models on standard benchmarks. Entity types provide a massive inductive bias that constrains what a relationship can possibly mean.

Baking Relational Awareness into Encoders

To capture deeper nuances without relying on explicit Named Entity Recognition (NER) pipelines, modern architectures bake relational awareness directly into the pre-training phase.

Changzhi Sun et al. developed the SPE (Span and span Pair Encoder) framework in Pre-training Entity Relation Encoder with Intra-span and Inter-span Information. Instead of relying on standard token-level masking (like BERT), SPE uses unsupervised objectives—such as contrastive span-pair matching and internal span permutations—to force the model to inherently represent how text segments interact.

Unlabeled Text: "The [acetylsalicylic acid] treats [acute migraines]."
                                 |                     |
                        [Span 1 Representation] <---> [Span 2 Representation]
                                                 |
                                    Contrastive Matching Loss

This structural awareness allows Large Language Models (LLMs) to perform relation extraction with zero human annotations. As shown by Yilmazcan Ozyurt et al. in Document-Level In-Context Few-Shot Relation Extraction via Pre-Trained Language Models, prompting pre-trained models in an in-context learning setup yields state-of-the-art document-level extractions. This bypasses error-prone preprocessing steps entirely, allowing the model's internal attention mechanisms to isolate and extract complex relationships autonomously.


Applications: From Text to Knowledge Graphs

When a system can extract relations without a predefined schema, it can transform massive, unstructured text dumps into structured knowledge networks on the fly.

For instance, Lixian Liu et al. introduced KGSS in Unsupervised Knowledge Graph Generation Using Semantic Similarity Matching, a framework that automatically constructs entire knowledge graphs from raw text by matching semantic similarities between open-ended entity pairs.

This paradigm has proven vital for:

By shifting the extraction problem from human-curated semantic classification to unsupervised structural discovery, schema-agnostic relation extraction provides the mathematical and algorithmic foundation required for machines to truly read the web.

Key Foundational Work

An Unsupervised Learning Method for Associative Relationships between Verb Phrases (2002) — contributed 1 concept elements to this direction.
Shipping Departments vs. Shipping Pacemakers: Using Thematic Analysis to Improve Tagging Accuracy (1992) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_open_information_extraction_2fca655a,
  title = {open information extraction},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/open-information-extraction/rdr_2fca655a745342f29e3e8f9728ef05cb},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_open_information_extraction_2fca655a_v1,
  title = {open information extraction},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/open-information-extraction/rdr_2fca655a745342f29e3e8f9728ef05cb/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake