Chinese has no spaces between words, so a named-entity tagger first has to guess where words begin and end. The best-known remedy is the word lattice: every substring that matches an external dictionary is added as an extra "word node" over the characters, and a neural model learns which nodes to trust. It works well on words the dictionary contains and not at all on words it lacks. A new company name, a slang term, or a transliterated person name gets no node. Worse, the dictionary words that do match around it are often wrong, because they straddle the true boundary of the unknown word. The entities the dictionary misses are exactly the ones a deployed extractor most needs to find.
We propose to open the lattice. Candidate spans come from three independent sources. The first is the dictionary. The second is structure induced without supervision from raw text of the target domain: character-level boundary statistics and shallow unsupervised chunks. The third is a small set of deterministic affix rules, such as organization and place suffixes. Each node carries a tag recording which source proposed it. Only dictionary nodes get a dictionary embedding. Nodes proposed by structure or rules are represented by composing their characters. In plain terms, we keep separate the evidence that "this is a word-shaped unit" and the evidence for "what this unit means," and we let the model learn how far to trust each kind of evidence.
Core Claim
Two separable ingredients drive the claim. First, the gain that explicit syntax gives on unseen entities, reported for gold constituent trees, can largely be recovered without any treebank by inserting structure-induced spans into a lattice. Second, those spans help only if their origin stays visible to the model, rather than being merged into the dictionary.
The first ingredient is motivated by Chinese NER with Height-Limited Constituent Parsing. That paper jointly predicts entities and shallow constituent trees of height at most 3 over a lattice encoder. It reports 76.67 F1 on OntoNotes 4.0 and a 6.34-point recall gain on out-of-vocabulary (OOV) entities, meaning entities whose words were never seen. It needs gold trees to train, however, which rules it out for new domains. We use shallow structure only as a source of candidate nodes, and we build that structure from raw text.
The second ingredient is what separates this work from the obvious move of mining new words and appending them to the dictionary. That appending baseline is a central comparison in our experiments. If appending does as well, the provenance claim is false.
Research Program
The backbone is the flat-lattice encoder of FLAT. FLAT turns the word–character graph into a flat sequence of spans. It relates any two spans through four head/tail distances (head-to-head, head-to-tail, tail-to-head, tail-to-tail), so it can accept any span as a node at no architectural cost. We reuse it unchanged. We also run a variant with a pretrained character encoder underneath, since a reviewer will reasonably ask whether any gain survives pretraining.
Each span node $s$ covering characters $c_i \dots c_j$ receives the representation
$$h_s = g(c_i,\dots,c_j) + \sum_{k} \mathbb{1}[s \in \mathcal{S}_k]\, e_k + \mathbb{1}[s \in \mathcal{S}_{\text{lex}}]\, w_s,$$
The terms are defined as follows. $g$ is a character-composition function (a small attention pool, with hashed character n-gram features in the spirit of HashFormers, which embeds arbitrary strings from a fixed parameter budget). $\mathcal{S}_k$ is the set of spans proposed by source $k$, and $e_k$ is a learned embedding for that source. $w_s$ is a dictionary word vector, which exists only when the span is in the dictionary. A span proposed by several sources collects all of its tags, so agreement between sources is directly visible to the model.
The induced sources are built from unlabeled text of the target domain only. The boundary source trains a character language model and proposes spans between points of high surprisal or high branching entropy. This follows the finding in Interpreting Word-Level Hidden State Behaviour of Character-Level LSTM Language Models that character models compress word-level information at implicit boundaries. The chunk source runs the cascaded probabilistic right-linear grammars of Simple Unsupervised Grammar Induction from Raw Text with Cascaded Finite State Models. That method reaches 39.8 unlabeled PARSEVAL F on the Chinese Treebank, and its induced noun-phrase units align with semantic units; we apply it over characters or boundary-source pseudo-words.
The rule source is a few dozen hand-written affix templates. This is the hybrid-extraction division of labor from systems such as OCR++: rules handle predictable shapes and the statistical model handles variance. Here, though, the rules only propose nodes and never make final decisions, which avoids the zero-recall failure of cascaded pipelines.
We cap candidate spans at a length and density budget, because FLAT's attention cost grows with the number of spans.
The analysis side has three parts. We measure how often each source proposes the gold span of an OOV entity. We measure how often dictionary matches cross OOV-entity boundaries. We also examine the collapse effect described by Murawaki et al. and Sornlertlamvanich et al.: an unknown word in unsegmented text mis-segments its neighbours too. We test this by scoring entities that sit next to OOV spans.
Baselines
The do-nothing reference is a character-only tagger with no dictionary. The file reports 88.81 F1 on MSRA for this setting, versus 93.18 for Chinese NER Using Lattice LSTM, which builds the dictionary lattice with gated recurrent cells. Any added source must beat the character baseline on OOV recall and must not lose ground on in-dictionary entities.
The strongest dictionary-only competitors are FLAT and SoftLexicon. FLAT reports 76.45 F1 on OntoNotes 4.0. SoftLexicon (Simplify the Usage of Lexicon in Chinese NER) pools dictionary matches into Begin/Middle/End/Single sets and reports beating Lattice LSTM by 1.76 points on OntoNotes. They are the strongest competitors because they isolate what our structure-based nodes add over dictionary nodes. Both run with the identical dictionary, the same character embeddings, and the same hyperparameter search budget.
The strongest syntax-aware competitor is the height-limited joint parser: 76.67 F1 on OntoNotes 4.0 and +6.34 OOV recall, which is the number to beat. Beating it means matching or exceeding 76.67 F1 in the same non-pretrained setting while using no gold trees. It also means achieving an OOV-recall gain over FLAT at least as large as 6.34 points. On a new domain without trees, it can only run with silver parses, so we also train it on trees from a parser trained on OntoNotes and applied to the target domain.
In the pretrained-encoder setting, the comparison is a pretrained character encoder with SoftLexicon features; no reported result was found on Lacuna for that configuration. On Weibo and Resume, the file reports only CAN-NER, a dictionary-free convolution-and-attention tagger, at 59.31 and 94.94 F1. Lacuna has no reported FLAT or SoftLexicon numbers for those sets, so we will run them ourselves.
Two ablations isolate the key ingredients. The first is the dictionary-append ablation: all induced and rule spans are added to the dictionary with no source tags and no split between composed and looked-up representations. The second drops each source in turn.
Why not just…?
Why not just mine new words from raw text and add them to the dictionary? Statistical new-word discovery is old, cheap, and plausibly captures most of the benefit, so this is the real threat. The dictionary-append ablation is built to settle it. We claim that provenance tags plus character composition beat appending, specifically on OOV-entity recall and on dictionary matches that cross entity boundaries. If appending matches us within noise, we will report that the value lies in the induced sources and not in keeping sources separate, and we will drop the separation claim.
Why not just use a strong pretrained character model and let it learn boundaries implicitly? The evidence in the file is mixed. CAN-NER notes it falls below feature-rich models on OntoNotes. FLAT's ablation shows that masking dictionary links hurts. None of that evidence involves large pretraining, and pretraining may absorb what gold syntax gave the height-limited parser. We therefore run every comparison both with and without a pretrained encoder. We treat a vanished gain under pretraining as a genuine negative result, not a detail to explain away.
Why not just train the height-limited parser on silver trees from an off-the-shelf parser? This keeps the joint-parsing design and removes the treebank requirement on paper. Parser errors under domain shift, however, are exactly what Dependency-based Semantic Role Labeling of PropBank documents degrading downstream semantics: its news-trained system drops on the out-of-domain Brown corpus. The cross-domain experiment tests this directly with silver trees in the target domain.
Proposed Experiments
In-domain experiments use OntoNotes 4.0, MSRA, Weibo, and Resume with standard splits. We report overall F1 and OOV-entity recall and precision. An entity counts as OOV if its surface string is absent from both the training entities and the dictionary.
Transfer experiments train on OntoNotes 4.0 and test on Weibo and Resume. The induced sources are built from target-domain raw text disjoint from the test set. We report a transductive variant, which induces on the unlabeled test sentences, separately and label it as such.
The decisive checks are four. First, on OntoNotes without gold trees, OOV recall must improve over FLAT by at least the 6.34 points the gold-tree parser achieved. Second, the tagged open lattice must beat the dictionary-append ablation on OOV recall and on boundary-crossing errors. Third, in transfer, the gap over silver-tree parsing and FLAT must widen relative to in-domain results. Fourth, the gains must survive, even if smaller, with a pretrained encoder.
Failure on the first check means induced structure cannot stand in for gold syntax. Failure on the second means the separation of form from meaning evidence is unnecessary. Either outcome is informative.
References
- Chinese NER with Height-Limited Constituent Parsing
- FLAT: Chinese NER Using Flat-Lattice Transformer
- HashFormers: Towards Vocabulary-independent Pre-trained Transformers
- Interpreting Word-Level Hidden State Behaviour of Character-Level LSTM Language Models
- Simple Unsupervised Grammar Induction from Raw Text with Cascaded Finite State Models
- OCR++: A Robust Framework For Information Extraction from Scholarly Articles
- Non-parametric Bayesian Segmentation of Japanese Noun Phrases
- Panel: The State of the Art in Thai Language Processing
- Chinese NER Using Lattice LSTM
- Simplify the Usage of Lexicon in Chinese NER
- CAN-NER: Convolutional Attention Network for Chinese Named Entity Recognition
- Dependency-based Semantic Role Labeling of PropBank