Traditional Named Entity Recognition (NER) has long relied on sequence labeling, where each token in a sentence is assigned a single tag (such as BIO). While effective for simple entities, this approach struggles with "nested" structures—cases where one entity exists inside another. For example, in the phrase "the Chinese embassy in France," a system might need to identify "France" as a Location and "Chinese embassy in France" as an Organization. Because a single token can only have one tag in a standard sequence, these overlapping boundaries create a structural bottleneck.
To solve this, a growing body of research has shifted from labeling tokens to classifying spans. By treating every possible contiguous sequence of words as a candidate entity, researchers have developed a unified framework that naturally handles both flat and nested entities.
From Tokens to Spans
The core intuition behind this shift is to treat the span, rather than the word, as the primary unit of analysis. In an exhaustive span-based model, the system enumerates all possible contiguous word sequences up to a predefined maximum length. As Mohammad Golam Sohrab and Makoto Miwa demonstrate in their Deep Exhaustive Model, this allows the model to evaluate each segment independently.
If a sentence has $N$ tokens, there are roughly $O(N^2)$ possible spans. For each span, the model generates a representation—often by concatenating the embeddings of the start and end tokens or using an attentive pooling mechanism—and passes it to a classifier. Because each span is classified independently, the model can assign a "Location" label to the sub-span "France" and an "Organization" label to the larger span "Chinese embassy in France" without any architectural conflict.
Efficient Global Scoring
Enumerating every possible span can be computationally expensive. To address this, researchers have moved toward "global" prediction methods. Jianlin Su and colleagues introduced the Global Pointer (GP) architecture, which treats NER as a multi-label classification task over a score matrix. In this setup, every entry $(i, j)$ in a matrix represents the probability that a span starting at index $i$ and ending at index $j$ is a specific entity type. This approach is not only efficient but also provides a unified way to handle nested entities as overlapping entries in the matrix.
Similarly, Juntao Yu and Massimo Poesio reformulated NER as dependency parsing. By using a biaffine scoring model to evaluate all possible start-and-end token pairs, they created a system that achieves state-of-the-art results across diverse datasets by treating entity recognition as a link-prediction problem between boundaries.
Beyond Simple Entities
The span-based paradigm extends naturally to more complex information extraction tasks. When entities are treated as discrete nodes, identifying the relationships between them becomes a matter of classifying pairs of spans.
- Relation Extraction: Kalpit Dixit and Yaser Al-Onaizan developed a span-level model that identifies entities and their relationships simultaneously. By representing text as independent spans, the model captures the global context of an entire phrase, which is often more informative for relation extraction than individual word features.
- Graph Representations: The DYGIE framework, introduced by Yi Luan and David Wadden, uses dynamic span graphs to capture interactions between entities. This allows the model to share information across overlapping spans and coreferent mentions, leading to more coherent extractions in dense, technical texts.
- Discontinuous Entities: In specialized fields like medicine, entities are sometimes fragmented (e.g., "productive... cough"). Bowen Yu and others have extended span logic to maximal clique discovery, where fragmented spans are treated as nodes in a graph, and the model searches for the most likely grouping of these fragments.
Why It Works
The success of exhaustive span classification lies in its ability to bypass the "one-label-per-token" constraint of sequence labeling. By evaluating spans as independent units, these models:
- Capture Multi-word Semantics: A span representation (like the one used in MGNER by Congying Xia) captures the meaning of a whole phrase, which is more robust than aggregating individual token tags.
- Resolve Structural Ambiguity: Nesting and overlapping are no longer "edge cases" but are inherent to the model's design.
- Facilitate Multi-tasking: As seen in SpanMlt, span-based architectures allow for joint learning of boundaries, types, and relations in a single end-to-end pass.
By moving away from the linear constraints of the sentence and toward a two-dimensional view of all possible substrings, these models provide a more flexible and powerful foundation for understanding complex language structures.