In Layman's Terms: Semi-Open Relation Extraction from Scientific Texts

Ruben Kruiper, Julian F. V. Vincent, Jessica Chen-Burger, Marc P. Y. Desmulliez, Ioannis KonstasACL 2020 · DOI

Scientific literature contains a wealth of knowledge, yet it remains largely inaccessible to non-experts—researchers from adjacent fields, engineers seeking biological inspiration, or the general public. The primary barrier is the complexity and density of scientific language. Information Extraction (IE) aims to bridge this gap by automatically distilling key findings into structured formats. However, traditional IE methods often struggle with the nuance and scale of scientific discourse. This paper introduces Semi-Open Relation Extraction (SORE), a hybrid framework designed to capture both the precision of narrow domain-specific relations and the broad context of open-domain interactions.

The Challenge: Precision vs. Breadth

Existing IE systems generally fall into two categories, both of which have significant drawbacks when applied to scientific texts:

  1. Narrow IE: These systems are highly accurate but restricted to a predefined, small set of relations (e.g., "is-a" or "part-of"). While precise, they often miss the "big picture" and the diverse interactions that characterize scientific discovery.
  2. Open IE: These systems attempt to extract any relation found in a sentence. However, scientific sentences are notoriously long and syntactically complex. On such data, Open IE systems frequently produce "hallucinations," uninformative trivialities, or fragmented tuples that lack necessary context.

Semi-Open Relation Extraction (SORE)

To address these issues, the authors propose SORE, which uses "central themes" of a paper as anchors to filter and organize broader information. In this study, the authors chose trade-offs (e.g., the balance between "safety" and "efficiency") as the anchor relation. Trade-offs are fundamental to fields like evolutionary biology and engineering, often defining the core problem space of a research paper.

Figure 1: Conceptual overview of SORE. Central relations like 'trade-offs' act as anchors to filter and contextualize diverse Open IE extractions.

The SORE Pipeline

The SORE pipeline operates in four distinct stages to refine noisy data into informative insights:

  1. Narrow Extraction: A specialized scientific IE system (SCIIE) identifies "central" relations, specifically trade-offs.
  2. Open Extraction: A standard Open IE system (OpenIE 5) extracts a wide array of relational tuples from the text.
  3. The SORE Filter: This is the system's core innovation. It uses the arguments from the high-precision "trade-off" extractions as semantic anchors. All extracted arguments are embedded into a vector space using ELMo and then clustered.
  4. Relevance Mapping: An Open IE extraction is only retained if its arguments belong to the same semantic clusters as the arguments of a central trade-off relation found in that specific text. This effectively uses the narrow system as a "relevance filter" for the noisy Open IE output.
Figure 2: The technical architecture of the SORE system, showing the transformation of text into embeddings and the subsequent clustering for relation identification.

The FOBIE Dataset

To evaluate their approach, the authors introduced FOBIE (Focused Open Biological Information Extraction). This new dataset consists of 1,548 sentences from 1,215 full-text biology papers, manually annotated for trade-off relations, their arguments, and modifiers. Compared to existing benchmarks like SCIENCEIE or SCIERC, FOBIE contains significantly longer and more complex sentences, with an average length of approximately 38 tokens, making it a rigorous testbed for scientific IE.

Key Results and Insights

The SORE approach demonstrated a remarkable ability to refine raw extraction data:

  • Noise Reduction: The system filtered out 65% of Open IE extractions. Manual verification confirmed that the majority of these discarded tuples were indeed erroneous or uninformative.
  • Improved Informativeness: The percentage of "informative" extractions—those judged useful by domain experts—rose from 29.25% to 35.00% after filtering.
  • Enhanced Sentence Utility: The proportion of sentences containing at least one informative extraction increased from 36.50% to 44.75%.

Beyond these metrics, SORE helps bridge the "jargon gap." By linking abstract, high-level terms (like "efficiency") to specific biological mechanisms (like "xylem" or "resilin") through semantic clustering, the system provides a more accessible entry point for non-expert readers.

Conclusion and Future Work

SORE represents a significant step toward making scientific literature more digestible. By treating Open IE as a "noisy candidate generator" and using Narrow IE to provide a "gold standard" for relevance, the framework achieves a balance between breadth and precision. While currently focused on biology and trade-offs, the SORE framework is designed to be generalizable. Future work could involve identifying central relations for other fields, such as "causality" in medicine or "composition" in materials science, and exploring more sophisticated clustering methods to further improve relevance mapping.

Summary updated

BibTeX
@inproceedings{kruiper2020layman,
  author = {Ruben Kruiper and Julian F. V. Vincent and Jessica Chen-Burger and Marc P. Y. Desmulliez and Ioannis Konstas},
  title = {{In Layman's Terms: Semi-Open Relation Extraction from Scientific Texts}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2020},
  doi = {10.18653/v1/2020.acl-main.137},
}
Figure