Teaching Domain Encoders Why a Span Is Nested

Nested entity recognition means finding named things inside other named things. In "胃痛" (stomach pain) the stomach is a body part and the whole phrase is a symptom. In "IL-2 receptor alpha chain" a protein name sits inside a larger protein name. The best systems for this task in medicine are compact encoders that were pre-trained further on medical text and then fitted with a head that scores every candidate span. They still fail on exactly the cases that make the task nested. On the Chinese medical benchmark CMeEE, the strongest such system gets 59.48% accuracy on nested entities against 71.75% on flat ones. It also routinely mixes up overlapping types such as symptom and disease (MRC-based Nested Medical NER with Co-prediction and Adaptive Pre-training). Its authors point to reasoning as the missing ingredient. The obvious ways to add reasoning are prompting a large language model or generating explanations at test time, and neither works well here. Frontier models prompted with a few examples trail small fine-tuned domain encoders by about 17 F1 on biomedical NER (En-ViMedNER). Generating text at test time is also slow.

We propose to use explanations only as training signal. For each gold pair of an inner and an outer entity, an LLM writes a short explanation of why the inner span has its type and why the enclosing span has a different one. The LLM sees the gold labels and the annotation guideline, so it explains a known answer rather than predicting one. During training, a small decoder attached to the domain encoder's span representations must reproduce that explanation. The decoder is thrown away afterwards, so inference cost is identical to the plain span classifier. The bet is that forcing span representations to carry the reason for a containment relation fixes the inner/outer confusion better than label supervision alone. We expect the effect to be largest when labeled data is scarce.

Core Claim

Explanations written for the known labels of nested pairs, and used as a training-only auxiliary target, will narrow the nested-versus-flat gap of domain-pretrained span classifiers. They will do so beyond what three simpler additions achieve on their own: the same explanation budget spent on static type definitions, rationales with scrambled content, or LLM pseudo-labels. What would be new is the supervision unit. The explanation is attached to a (inner span, outer span) pair and describes their relationship. It is not attached to a sentence or to a single label.

This separates the proposal from its nearest neighbours. MRC-CAP (MRC-based Nested Medical NER) phrases each entity type as a natural-language question, such as "find all body parts." That text is fixed per type and never explains any particular instance. The explanation-guided work in the file, EGM-PM with its ETCM active-learning scheme (Enhancing the Interpretability of NLI Models), works on sentence-pair classification and consumes explanations at prediction or selection time. We move explanations down to span pairs and use them only in training. Earlier nested NER models get their structure from somewhere else. MHGAT (A Multi-Scale Heterogeneous Graph Attention Network for Nested NER) uses parse trees, and EIoU-EMC (EIoU-EMC) uses a geometric boundary loss. Both are orthogonal to ours and could be stacked with it. BioBART (BioBART) generates entity strings with no intermediate justification, and it supplies one of our backbones.

Research Program

The base model is a domain-pretrained encoder with a biaffine span head. A biaffine head scores every (start, end) token pair for each entity type, in the style of the exhaustive span models of Deep Exhaustive Model for Nested NER and the biaffine branch of MRC-CAP.

The explanation corpus is built from the training split only. For every gold nested pair, and for a sample of "confusable" spans the base model mislabels on held-out folds, an LLM receives three things: the sentence, the two spans with their gold types, and the relevant guideline entry. It writes one or two sentences explaining the containment and the type distinction. Direction 1's main known weakness is plausible but ungrounded or tautological explanations, so we filter in two ways. First, an explanation is kept only if a separate LLM call, given the sentence and the explanation but not the labels, can recover both gold types. Second, explanations that merely restate the type names are removed by a lexical-overlap rule.

In training, we mean-pool the encoder states over each span to get span vectors $h_{\text{in}}$ and $h_{\text{out}}$. A two-layer transformer decoder conditioned on $[h_{\text{in}}; h_{\text{out}}]$ must generate the explanation $r$. The objective is

$$\mathcal{L} = \mathcal{L}_{\text{span}} + \lambda \, \frac{1}{|P|}\sum_{(i,o,r)\in P} -\log p_\theta(r \mid h_i, h_o),$$

where $\mathcal{L}_{\text{span}}$ is the usual span-classification loss, $P$ is the set of explained pairs in the batch, and $\lambda$ weights the auxiliary term. The key design choice is that the decoder sees only the two span vectors, never the full sentence. That forces the reason for the distinction into the representations the span classifier actually uses. This follows the Direction 1 idea that a generative rationale loss penalizes shortcut features that a label loss alone lets through. GRACE gives related evidence: optimizing over generated reasoning improves encoder representations.

Domain pre-training enters as a controlled factor rather than a given. We cross two backbones, general versus domain-pretrained (Chinese medical NEZHA-style for CMeEE, BioBERT or the BioBART encoder for GENIA), with two conditions, explanation loss versus none. We also vary the amount of training data across 1%, 2%, 10% and 100%, matching the low-resource protocol of EIoU-EMC. This answers a question the literature leaves open. Do explanations mostly substitute for domain pre-training, so that the gain is larger on general backbones? Or do they need domain grounding to be usable, so that the gain is larger on domain backbones? The second outcome would be expected if general encoders cannot represent the terms the explanations mention.

Baselines

The naive reference is the same domain encoder with the biaffine span head and no auxiliary loss. It is trained with identical data, schedule and hyperparameter search. Every gain is measured against it.

The strongest existing method on the main benchmark is MRC-CAP, which reports 77.04 F1 on CMeEE V2 and 69.42 F1 on V1, with nested/flat accuracy of 59.48%/71.75% (MRC-based Nested Medical NER). It is the strongest competitor because it already combines domain continual pre-training with a nested-capable dual head, and it already feeds natural-language text about types to the model. We will re-run it under our splits and tuning budget, using the released backbone if one is available and otherwise our own continually pre-trained encoder. We report our reproduction next to the published 77.04. We then add our explanation loss on top of MRC-CAP itself, so the comparison is additive rather than a contest between architectures. Beating it means a gain in nested-span F1 of at least one point over the reproduced MRC-CAP, averaged over five seeds and outside seed variance, with overall F1 not dropping.

On GENIA, the strongest number in the file is MHGAT's 82.2 F1 (MHGAT). It uses a general SpanBERT backbone, so we treat it as a reference point rather than a like-for-like competitor. We found no reported overall GENIA result on Lacuna for BioBART or for BioBERT with EIoU-EMC. On ICEM, EIoU-EMC reports 56.32 F1 (EIoU-EMC). We will re-run it and combine it with our loss, since the two target different failures, boundaries versus types.

Three ablations isolate the key ingredient. The first is the static-definition control: the same decoder generates the guideline definition of each type instead of the pair-specific explanation. It uses the same number of generated tokens. The second is the shuffled control: explanations are reassigned to random pairs of the same type combination, which keeps the regularizing effect of generation but breaks the instance content. The third is the unfiltered control, which drops the faithfulness filter. All three use the same LLM budget, the same λ search and the same compute.

Why not just…?

Why not just prompt a frontier LLM with chain-of-thought? The file's evidence goes against it for this task. Gemini 3.1 Flash scored 36.47 F1 three-shot versus 52.70 for a fine-tuned ViPubMedDeBERTa on En-ViMedNER (En-ViMedNER). GPT-4 was clearly beaten by MRC-CAP on CMeEE. Autoregressive LLM labeling was also about 9x slower than direct decoding in DIRECT. We still include a few-shot chain-of-thought run of the same LLM that writes our explanations. This shows how much the LLM knows when it lacks the answer, compared with how useful it is once shown the answer.

Why not just put richer type definitions or guidelines into the MRC queries? This is the real threat. Much of the symptom/disease confusion might be fixed by better static text, with no per-instance explanation at all. The static-definition ablation and a richer-query MRC-CAP variant test this directly. If either matches the pair-specific explanations on nested-span F1, we will conclude that the value lies in type knowledge rather than instance reasoning. We will report it that way and not claim the mechanism.

Why not just distill the LLM's labels, or self-train on unlabeled domain text? Pseudo-labels transfer the LLM's answers, and the evidence above says those answers are poor in this domain. Our explanations are written given the gold answers, so they transfer the LLM's descriptive knowledge without its prediction errors. To make the test fair, we add a matched-budget control that spends the same LLM calls on pseudo-labeling unlabeled CMeEE-style sentences.

Why not just continue domain pre-training longer? Domain pre-training is a real lever, and the 2x2 design measures it directly. The nested gap persists in MRC-CAP despite its adaptive pre-training, which suggests the missing piece is not terminology alone. If extra pre-training on the backbone closes the gap as much as our loss does at equal compute, the proposal fails.

Decisive Checks

The main experiments run on CMeEE V2, where 31.79% of entities are nested and symptoms nest about 66% of the time, on CMeEE V1, on GENIA, and on ICEM for the long-tail industrial setting.

Beyond overall micro-F1, we report nested-span F1, flat-span F1 and their gap. We report inner and outer accuracy separately. We report the symptom↔disease confusion rate from the confusion matrix, and per-category F1 for rare types such as GENIA RNA, where EIoU-EMC reports a 1.94-point gain.

The proposal is supported under three conditions. First, the explanation loss narrows the nested/flat gap on CMeEE V2 relative to reproduced MRC-CAP. Second, it beats both the static-definition and shuffled controls, with the margin growing as training data shrinks toward 1%. Third, the symptom/disease confusion rate falls. It is refuted if the shuffled control matches the full method, because the gain would then be generic generation regularization, or if the static-definition control matches it.

A secondary check tests whether the effect is truly driven by explanations. The pair-specific explanations from a stronger and a weaker LLM should produce gains that track the pass rate of the faithfulness filter. Inference latency and parameter count should be identical to the baseline, because the decoder is removed after training.