The Limitations of Fixed Length Constraints in Span Extraction

49 concepts48 papers1 proposal

In natural language processing, moving from token-level sequence labeling (such as BIO tagging) to span-based modeling was a major conceptual breakthrough. Instead of forcing a model to make independent decisions for every individual word, span-based architectures evaluate entire phrases holistically. This paradigm shift makes it significantly easier to extract nested entities, resolve overlapping coreferences, and capture multi-word relationships directly.

However, this elegance comes with a severe computational catch. If a sentence contains $N$ tokens, the number of possible candidate spans scales quadratically as $O(N^2)$. For tasks involving pairs of spans—such as relation extraction—the search space explodes even further.

To keep these models computationally tractable, researchers almost universally apply a pragmatic engineering heuristic: a fixed maximum span length constraint ($L$). By ignoring any candidate span longer than $L$ tokens, the search space is reduced from a quadratic explosion to a manageable, linear $O(NL)$ complexity. Yet, as a wide body of literature demonstrates, this artificial ceiling introduces a fundamental failure mode across diverse NLP tasks.

The Combinatorial Explosion and the Fixed-Length Fix

To understand why the fixed-length constraint is so pervasive, consider the mechanics of an exhaustive span classifier. As noted by Shuhui Wu et al. in their work on nested named entity recognition (Propose-and-Refine: A Two-Stage Set Prediction Network for Nested Named Entity Recognition), checking every possible token sequence quickly becomes prohibitive as sentence length grows.

To prevent out-of-memory errors during the candidate generation phase, models must prune the search space before heavy classification networks are applied. Bin Ji et al. demonstrate this standard trade-off in Win-Win Cooperation: Bundling Sequence and Span Models for Named Entity Recognition, where a hard length-based threshold is enforced during segment enumeration. This constraint successfully reduces the number of required classifications, but it creates an absolute blind spot: the model becomes mathematically incapable of seeing any semantic unit that exceeds the predefined window size.

The Blind Spots of the Hard Ceiling

The core flaw of a fixed-length constraint is that natural language does not adhere to arbitrary token caps. When a model encounters highly descriptive, technical, or formal language, the heuristic fails.

Named Entity Recognition and Information Extraction

In specialized domains like legal, medical, or scientific literature, entity names are frequently long and complex. Enwei Zhu et al. (Deep Span Representations for Named Entity Recognition) point out that predefined span limits leave models incapable of retrieving exceptionally long citations, titles, or legal entities.

Similarly, when extracting information from wet lab protocols, Mohammad Golam Sohrab et al. (mgsohrab at WNUT 2020 Shared Task-1: Neural Exhaustive Approach for Entity and Relation Recognition Over Wet Lab Protocols) observe that while exhaustive span enumeration handles nested relations well, the maximum length heuristic cuts off highly descriptive entity mentions that fall outside the window.

Coreference Resolution and Entity Linking

Coreference resolution requires identifying when different text segments refer to the same real-world entity. Noun phrases can be heavily modified by adjectives and relative clauses, stretching their length significantly. Rui Zhang et al. (Neural Coreference Resolution with Deep Biaffine Attention by Joint Mention Detection and Mention Clustering) show that restricting candidate spans to a fixed maximum length systematically fails to capture these long, descriptive noun phrases.

When these mentions are truncated or missed entirely, downstream document-level consistency breaks down, a vulnerability highlighted by Johannes Deleu et al. (Towards Consistent Document-level Entity Linking: Joint Models for Entity Linking and Coreference Resolution).

Sentiment and Attribute Extraction

In aspect-based sentiment analysis and e-commerce extraction, targets are rarely single words. Longtao Huang et al. (A Span-based Joint Model for Opinion Target Extraction and Target Sentiment Classification) demonstrate that fixed-length span enumeration cannot identify long, descriptive targets in complex sentences.

This issue is compounded in product attribute extraction; Yan Liang et al. (Ask-and-Verify: Span Candidate Generation and Verification for Attribute Value Extraction) note that fixed-length windowing fails to capture detailed usage instructions or multi-sentence descriptions commonly found in e-commerce catalogs.

Structural and Syntactic Truncation

The damage caused by fixed-length constraints isn't limited to missing isolated entities; it also disrupts the modeling of large-scale sentence structure.

In statistical and hierarchical machine translation, models rely on extracting translation rules over text spans. Adrià de Gispert et al. (Effective Incorporation of Source Syntax into Hierarchical Phrase-based Translation) explain that strict limits on span length prevent models from capturing long-distance reordering and complex syntactic structures. This constraint is often baked deeply into grammar extraction frameworks, such as the one analyzed by Paul Baltescu et al. (A Fast and Simple Online Synchronous Context Free Grammar Extractor), where heuristic pruning assumes that long-distance relationships contribute minimally to translation quality—an assumption that fails on complex linguistic structures.

A similar bottleneck occurs when integrating syntactic parsing directly into language modeling. Hao Peng et al. (PaLM: A Hybrid Parser and Language Model) show that imposing a maximum limit on span length restricts a model's ability to represent long-range dependencies, causing it to miss large syntactic constituents entirely.

Even in token-starved domains like Chinese word segmentation, where words are composed of characters without explicit spacing, length caps cause systematic errors. Deng Cai (Neural Word Segmentation Learning for Chinese) and Xiao Sun 0003 (Detecting New Words from Chinese Text Using Latent Semi-CRF Models) both demonstrate that predefined segment limits prevent the detection of unusually long technical terms, out-of-vocabulary words, or complex idioms, forcing an unnatural trade-off between search tractability and linguistic accuracy.

Fixed-Length Span Enumeration (Max Length L = 4):
[The Supreme Court of] [the United States ruled] ...
                      ^
                      Model cannot evaluate the full 7-token span:
                      "The Supreme Court of the United States"

Evolving Beyond the Fixed Window

Recognizing that hard length limits fundamentally cripple a model's ability to generalize, researchers have developed alternative paradigms to preserve the benefits of span-level modeling without the computational penalty of exhaustive enumeration.

1. Boundary-Based Representations

Instead of explicitly enumerating and encoding every possible intermediate span, models can focus entirely on span boundaries. Yuval Kirstain et al. (Coreference Resolution without Span Representations) introduced a minimalist framework that eliminates explicit span representations. By utilizing bilinear functions over only the start and end tokens of a span, their approach bypasses manual length-based filtering entirely, drastically reducing memory usage while naturally accommodating spans of any length.

2. Coarse-to-Fine (Propose-and-Refine) Networks

Rather than evaluating all spans equally, a model can use a multi-stage pipeline to filter candidates dynamically. The architecture proposed by Shuhui Wu et al. (Propose-and-Refine: A Two-Stage Set Prediction Network for Nested Named Entity Recognition) first generates coarse, length-agnostic entity proposals and then refines them using hierarchical features. This allows the model to break through the traditional length cap without suffering from a combinatorial explosion.

By shifting from rigid, fixed-window enumeration to flexible boundary tracking and coarse-to-fine filtering, modern architectures are successfully removing the artificial ceilings that long restricted span-based extraction.

Key Foundational Work

Automatic Extraction of New Words from Japanese Texts using Generalized Forward-Backward Search (1996) — contributed 1 concept elements to this direction.
Towards Automatic Extraction of Monolingual and Bilingual Terminology (1994) — contributed 1 concept elements to this direction.
History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 47 papers
  2. Version 1 · 2026-10-04 · 58 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_fixed_length_span_constraint_7d51f255,
  title = {fixed-length span constraint},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/fixed-length-span-constraint/rdr_7d51f255c0ea4c259e90b957a21db01a},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 2, frozen text):

@misc{lacuna_fixed_length_span_constraint_7d51f255_v2,
  title = {fixed-length span constraint},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/fixed-length-span-constraint/rdr_7d51f255c0ea4c259e90b957a21db01a/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake