Abstract
Current state-of-the-art models for named entity recognition (NER) are neural models with a conditional random field (CRF) as the final layer. Entities are represented as per-token labels with a special structure in order to decode them into spans. Current work eschews prior knowledge of how the span encoding scheme works and relies on the CRF learning which transitions are illegal and which are not to facilitate global coherence. We find that by constraining the output to suppress illegal transitions we can train a tagger with a cross-entropy loss twice as fast as a CRF with differences in F1 that are statistically insignificant, effectively eliminating the need for a CRF. We analyze the dynamics of tag co-occurrence to explain when these constraints are most effective and provide open source implementations of our tagger in both PyTorch and TensorFlow.
Introduction
In Named Entity Recognition (NER), tokens are typically labeled using structured schemes like IOBES (Begin, Inside, End, Single, Outside). These schemes impose strict transition rules; for instance, an "Inside" tag cannot follow an "Outside" tag without an intervening "Begin" tag. The standard approach to enforcing these rules is the Conditional Random Field (CRF) layer. While effective at learning transition probabilities, CRFs introduce significant computational overhead ($O(NT^2)$ complexity), complicate implementation, and substantially increase training time.
This paper explores whether the global coherence provided by a CRF is redundant in the era of high-capacity neural models like biLSTMs and Transformers. We propose Constrained Decoding as a lightweight, efficient alternative.
Methodology
Our approach decouples the learning of features from the enforcement of structural rules:
- Training with Cross-Entropy: Instead of a CRF, we train a standard bidirectional LSTM (biLSTM) using a simple token-level cross-entropy loss. This treats each token as an independent classification task, which is $O(N)$ and highly parallelizable, leading to much faster training cycles.
- Inference with a Constraint Mask: At test time, we still utilize the Viterbi algorithm to find the optimal sequence. However, rather than using a transition matrix learned from data, we apply a heuristic mask. This mask assigns a massive negative penalty to any transition that is "illegal" according to the IOBES scheme (e.g., "O" to "I-LOC"), effectively acting as a hard rule-enforcer.
Experimental Results
We evaluated the performance of Constrained Decoding against standard CRF models across several benchmarks, including CoNLL 2003, WNUT-17, OntoNotes, and Snips.
Efficiency and Speed
The constrained model demonstrated a dramatic improvement in efficiency, training in just 51.2% of the time required by an optimized, batched CRF. This speedup translates directly to environmental benefits, with the constrained model producing only 65% of the $CO_2$ emissions of its CRF counterpart. Furthermore, constrained models tended to converge faster, requiring an average of 60.6 epochs compared to 72.4 for the CRF on the CoNLL dataset.
Accuracy
As shown in the results table, the difference in F1 scores between the CRF and Constrained Decoding was statistically insignificant for most datasets. On CoNLL, the CRF achieved a mean F1 of 91.61 compared to 91.44 for the constrained model. Similar parity was observed for WNUT-17 and Snips.
Analysis of Dataset Dynamics
OntoNotes proved to be the exception, where the CRF significantly outperformed the constrained model (87.43 vs 86.13). To understand this, we analyzed the "strictly dominated" tokens in each dataset—tokens where the tagging scheme leaves only one valid choice based on the previous tag.
Datasets like CoNLL have a high percentage of strictly dominated tokens (71.2% for IOBES), meaning the hard constraints do most of the work. In contrast, OntoNotes has far fewer strictly dominated tokens (15.9%), requiring the model to rely more on the learned distributional probabilities of transitions (e.g., how likely a "Person" entity is to follow a "Location") that only a CRF can provide.
Conclusion
Constrained Decoding offers a compelling trade-off for NER tasks. By replacing the CRF with a simple cross-entropy loss and a heuristic inference mask, researchers and practitioners can achieve nearly identical accuracy with a 2x speedup in training. While the CRF remains superior for complex datasets with low structural dominance like OntoNotes, the efficiency and reduced carbon footprint of constrained decoding make it an ideal choice for most production environments and rapid iteration cycles.
Summary updated