Calibrating Document-Local Boundaries for Contextual Chunk Retrieval

Yuanxiang Wang, Hui Jiang, Da Zhu, Zhihua ChangarXiv 2026 · arXiv

TL;DR

The paper introduces DcLBC, a framework that sharpens the boundaries between semantically similar chunks within a single document. By combining adaptive loss scaling with chunk-aware attention routing, it significantly improves contextual retrieval accuracy across various encoder architectures without sacrificing general performance.

What Problem Does This Paper Address?

Modern long-document retrieval often uses "Late Chunking," where a whole document is encoded before being split into smaller units (chunks). This is great for preserving context—like knowing that a "he" in paragraph three refers to a person named in paragraph one.

However, this shared context creates a "sibling confusion" problem. Because chunks from the same document share the same global information, they look extremely similar to a retrieval model. The authors found that in over 82% of cases where a model failed to rank the correct chunk first, the "winner" was actually a sibling chunk from the same document. Standard training methods use a fixed "temperature" for their loss functions, which fails to provide a strong enough signal to resolve these fine-grained local confusions.

Methods

The authors propose Document-Chunk Local Boundary Calibration (DcLBC), which consists of two main components:

  1. Adaptive Margin Scaling (AMS): Instead of using a one-size-fits-all temperature parameter during training, AMS dynamically adjusts the loss scale for each query-document pair. It measures the margin between the "gold" (correct) chunk and its hardest sibling. If they are too close, it increases the scaling factor to "sharpen" the loss, forcing the model to focus its optimization effort on distinguishing that specific difficult pair.
  2. Chunk-Aware Key Routing (CKR): Standard Transformer encoders are "chunk-blind"—they don't know where one retrieval unit ends and another begins until the final pooling step. CKR injects this boundary information directly into the attention mechanism. It computes a "routing vector" for each chunk and adds it as an offset to the Keys in the final layers. This allows tokens to "know" which chunk they belong to and adjust their attention patterns accordingly.
Figure 2: Overview of DcLBC. CKR redistributes attention across chunks, while AMS adaptively scales within-document scores.

Datasets

The models were evaluated on two primary benchmarks:

  • ConTEB (Contextual Text Embedding Benchmark): An 8-task benchmark specifically designed to test retrieval that requires document context (e.g., resolving coreferences or specific dates). It includes tasks like SQuAD, NQA, and specialized datasets for Football and Cities.
  • NanoBEIR: A subset of the standard BEIR benchmark used to ensure that improvements in contextual retrieval don't degrade performance on standard, context-independent passage retrieval.

Key Insights and Ideas

  • The Entropy of Hard Negatives: The authors mathematically demonstrate that increasing the scaling factor ($\kappa$) monotonically decreases the entropy of the negative allocation. In simpler terms: higher scaling forces the model to stop "worrying" about easy negatives and put all its "effort" into the hardest, most confusing ones.
  • Complementary Roles: AMS and CKR tackle the problem from different angles. AMS is an optimization-side fix (changing how we train), while CKR is a representation-side fix (changing how the model processes information). Ablation studies showed that their combination yields a "positive interaction," performing better than the sum of their individual gains.
  • Dynamic Calibration: Training diagnostics show that AMS successfully maintains higher scales for difficult queries throughout the training process, providing a persistent "focused update" direction.
Figure 3: Query margins and adaptive scales during DcLBC training. Solid: example queries; dashed: training means.

Key Results

DcLBC consistently outperformed the previous state-of-the-art method (InSeNT) across multiple model backbones:

  • ModernBERT-large: Achieved a 78.61% NDCG@10 on ConTEB (a +1.91 percentage point gain) and 60.56% on NanoBEIR (+0.81 pp gain).
  • ModernColBERT: Saw the largest jump in contextual retrieval, with a +2.45 pp improvement on ConTEB.
  • Efficiency: Importantly, AMS introduces no overhead at inference time, and CKR adds only a tiny, shared bottleneck layer, making the improvements practically "free" for deployment.
Table 1: ConTEB and NanoBEIR NDCG@10 (%). Parentheses give base-model parameters in millions (M); bold/underline: best/second best. “–” indicates no additional post-training in this study.

Limitations and Future Work

The authors noted that the benefits of CKR are somewhat sensitive to configuration; their "layer coverage" tests showed a non-monotonic pattern, meaning that simply adding more routed layers doesn't always lead to better results. Finding the optimal "compact" configuration remains an empirical challenge.

While the paper focuses on bidirectional encoders, the success of these local boundary calibration techniques suggests they could be adapted for causal (decoder-only) LLM encoders, which currently struggle more with context propagation in retrieval tasks.

Summary updated

Abstract

Late Chunking encodes an entire document before pooling at chunk boundaries, preserving document context in local retrieval units. Chunk-level post-training improves context-dependent retrieval, but shared context can make semantically similar same-document negatives difficult to distinguish. We study the local decision boundary between a gold chunk and its hardest sibling. Starting from InSeNT, we analyze InSeNT's within-document gradients and show that the entropy of the conditional negative allocation decreases monotonically with the scaling factor at fixed logits. We propose a two-part framework for calibrating document-local chunk boundaries. The first component adapts the discriminative scale from within-document margins, while the second exposes chunk partitions to attention during document encoding. Experiments on ConTEB and NanoBEIR show that the resulting model improves contextual chunk retrieval across the evaluated encoder settings while retaining competitive passage-level retrieval quality; ablations further characterize the distinct roles of adaptive scaling and chunk-aware routing.
BibTeX
@misc{wang2026calibrating,
  author = {Yuanxiang Wang and Hui Jiang and Da Zhu and Zhihua Chang},
  title = {{Calibrating Document-Local Boundaries for Contextual Chunk Retrieval}},
  year = {2026},
  eprint = {2611.12097},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.12097},
}
Figure