Span Level Representations for Structured Information Extraction

This direction is merged into another direction: quadratic comparison complexity. Its last version stays readable and citable here.

Version 1 of 1170 papers187 conceptsfrozen 2026-10-04

Traditional Natural Language Processing treats text as a sequence of individual tokens. To identify a person's name or a chemical compound, we typically assign a label to each word—a process known as sequence labeling. However, this word-by-word approach struggles with "nested" structures, where one entity exists inside another (e.g., identifying "IL-2R alpha" as a protein within the larger "IL-2R alpha gene").

To solve this, researchers have shifted toward span-level modeling. Instead of labeling tokens, these models evaluate entire subsequences (spans) of text as single units. This shift allows models to capture the internal semantics of a phrase and naturally handle overlapping or nested entities.

The Exhaustive Search Problem

The most direct way to implement span-level extraction is to look at every possible subsequence in a sentence. In Deep Exhaustive Model for Nested Named Entity Recognition, Mohammad Golam Sohrab and Makoto Miwa demonstrated that by enumerating all spans and classifying them directly, models could outperform traditional sequence labelers on complex biomedical data.

However, this exhaustive approach introduces a significant computational bottleneck. For a sentence of length $n$, there are $O(n^2)$ possible spans. If a model also tries to identify relations between these spans—such as "who works for whom"—the search space explodes to $O(n^4)$. As noted by Wei Xu and colleagues in their work on SpanRel, this combinatorial explosion makes naive span models difficult to scale to long documents.

Pruning the Search Space

To make span-level extraction practical, the research has moved toward "Locate and Label" architectures. Rather than scoring every possible span, these models first identify high-probability "entity-rich" regions and then refine them.

Yongliang Shen and Xinyin Ma proposed a two-stage framework in Locate and Label: A Two-stage Identifier for Nested Named Entity Recognition that borrows from object detection in computer vision. By first proposing candidate spans and then classifying them, they significantly reduce the number of redundant calculations. Similarly, Lefei Wu and Qinghua Zheng introduced boundary regression, treating entity boundaries as continuous values that can be adjusted, much like bounding boxes in an image.

Beyond Entities: Relations and Graphs

The power of span representations extends beyond simple entity recognition. By treating spans as nodes in a graph, models can perform multiple extraction tasks simultaneously.

The DYGIE framework, developed by Yi Luan, David Wadden, and others in A general framework for information extraction using dynamic span graphs, uses this graph-based approach. It builds representations for spans and then dynamically updates them based on the relations and coreference links identified between them. This allows the model to use the fact that two spans are related to better understand what those spans actually are.

Efficiency through Language Modeling

Recent work has also explored reformulating span extraction as a language modeling problem to avoid explicit enumeration. Linyang Li and Tao Gui introduced EntLM in Template-free Prompt Tuning for Few-shot NER, which predicts "pivot words" at entity positions. This approach achieved a staggering 1930x speedup over template-based span search by moving the complexity from the search space into the model's internal vocabulary predictions.

Why It Works

The intuition behind the success of span-level models is twofold:

  1. Boundary Integrity: By looking at a span as a whole, the model can ensure that the start and end of an entity make sense together, reducing the "fragmented" predictions common in word-by-word tagging.
  2. Contextual Pooling: Span-level models often use max-pooling or attention over the tokens within a span. This creates a single vector that summarizes the entire phrase, making it easier for the model to compare that phrase against a dictionary or a relation template.

While the computational cost remains a challenge, the transition from token-level to span-level representations has become the standard for high-precision, structured information extraction in complex domains.

Papers in this version

Showing 60 of 170 papers.

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_span_level_representations_for_structure_0970f670,
  title = {Span Level Representations for Structured Information Extraction},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/span-level-representations-for-structured-information-extraction/rdr_0970f670b69341d1b9bfa786056ba088},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 1, frozen text):

@misc{lacuna_span_level_representations_for_structure_0970f670_v1,
  title = {Span Level Representations for Structured Information Extraction},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/span-level-representations-for-structured-information-extraction/rdr_0970f670b69341d1b9bfa786056ba088/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}