Traditional Named Entity Recognition (NER) treats text as a flat sequence of tokens. Under standard tagging schemes like BIO (Begin, Inside, Outside), each token is assigned exactly one label. For instance, in the phrase "New York City," a model tags "New" as B-LOC, and "York" and "City" as I-LOC.
This one-to-one mapping relies on a fundamental assumption: entity boundaries are mutually exclusive. However, natural language—especially in specialized domains like medicine, law, or technical literature—frequently violates this assumption. Consider the phrase:
"...a Pennsylvania radio station..."
Here, "Pennsylvania" is a geographic location (GPE), but the entire phrase "Pennsylvania radio station" functions as a single organizational entity (ORG). If a token-level classifier assigns B-GPE to "Pennsylvania," it cannot simultaneously assign it B-ORG without violating the single-label constraint. As noted by Greenberg et al. in their work on Marginal Likelihood Training of BiLSTM-CRF for Biomedical Named Entity Recognition from Disjoint Label Sets, this structural limitation forces models to omit nested or embedded entities, severely hurting recall in dense information extraction tasks.
The Shift to Span-Based Architectures
To resolve this conflict, researchers shifted the problem framing from token classification to span classification. Instead of asking "What is the label of this word?", span-based models ask "Is this specific sequence of words a valid entity?"
In Deep Exhaustive Model for Nested Named Entity Recognition, Sohrab et al. proposed a straightforward solution: enumerate every possible text segment (span) up to a maximum length within a sentence and classify each independently.
Sentence: [a] [Pennsylvania] [radio] [station]
Spans evaluated:
Length 1: [a], [Pennsylvania] (GPE), [radio], [station]
Length 2: [a Pennsylvania], [Pennsylvania radio], [radio station]
Length 3: [a Pennsylvania radio], [Pennsylvania radio station] (ORG)
By evaluating spans directly, the model decouples the prediction of the inner entity (GPE) from the outer entity (ORG). Each span receives its own representation—often generated by pooling the token embeddings within its boundaries—allowing the classifier to assign labels to overlapping regions without any architectural conflict.
Graph and Table Representations
While exhaustive span enumeration is conceptually simple, it can be computationally expensive since the number of candidate spans grows quadratically with sentence length. To mitigate this, researchers introduced alternative structural representations that map overlapping entities into multi-dimensional spaces.
One prominent approach is hypergraph-based modeling. As demonstrated by Muis et al. in Learning to Recognize Discontiguous Entities, text can be represented as a structural graph where nodes represent word gaps and edges represent entity mentions. By allowing multiple paths or overlapping hyperedges to coexist over the same sequence of tokens, hypergraphs can elegantly encode nested and even discontiguous entity structures without exploding the search space.
Another paradigm is table filling. Instead of a linear sequence of tags, the sentence is projected onto a two-dimensional matrix where rows and columns correspond to token positions. The cell at intersection $(i, j)$ represents the relationship or entity span stretching from token $i$ to token $j$. Gupta et al. explored this in Table Filling Multi-Task Recurrent Neural Network for Joint Entity and Relation Extraction, demonstrating that treating entity extraction as a matrix-filling problem naturally accommodates overlapping spans, as different layers or cell values can represent distinct entity boundaries simultaneously.
Generative and Multi-Task Frameworks
With the rise of sequence-to-sequence architectures, another intuitive solution has emerged: transforming the nested extraction task into a text generation problem. Instead of predicting boundaries via classification layers, generative models rewrite the input sentence into an augmented natural language format that explicitly includes the hierarchical tags.
For example, a generative model might output:
[ORG: a [GPE: Pennsylvania] radio station]
As highlighted by Alotaibi et al. in muNERa at WojoodNER 2024: Multi-tasking NER Approach, this approach completely bypasses the constraints of encoder-only sequence labelers. By leveraging an autoregressive decoder, the model can generate arbitrary tree-like structures, making it highly effective at capturing fine-grained, multi-layered entity hierarchies.