When a document is too long for a language model, it gets cut into chunks. Usually the cut falls every $N$ tokens, and neighbouring chunks share a fixed overlap, often 50%, so that no sentence is stranded at an edge. Two problems follow. First, $N$ tokens is not the same amount of text in every language. The same fixed-token cut falls at a different point in the meaning of an English report than in its Arabic or Russian translation, because some languages need more tokens per word. Second, a uniform overlap spends the same extra compute at every seam. Seams that happen to land between two topics need no overlap. Seams that land mid-argument need a lot.
This note proposes a simple change and a controlled test of it. Boundaries are placed at "cohesion valleys": points where the similarity between the text just before and just after dips relative to nearby peaks. The windows used to find those valleys are measured in words or sentences, never tokens. Overlap is then spent only where a seam could not be placed in a deep valley. The claim is about compute. At a fixed total overlap budget, cutting at valleys and giving overlap only to shallow seams should beat both fixed-token chunking at the same budget and fixed-token chunking at a 50% budget. The test holds content constant by using parallel documents in six languages, so any change in the gap can be traced to how each language is tokenized.
Core Claim and What Is Actually New
The detector is not new. TextTiling, from Multi-Paragraph Segmentation of Expository Text, compares adjacent word blocks with cosine similarity and scores each dip by its "depth": how far it sits below the peaks on either side. A texttiling based approach to topic boundary detection in meetings showed this relative depth survives the high baseline similarity of jargon-heavy meetings. M³Seg replaced cosine with a learned mutual-information gap and reached $P_k = 0.248$ on AMI. ($P_k$ is the probability that two nearby positions are wrongly judged to be in the same or different segments; lower is better.) We reuse these as the boundary scorer.
What prior work has not done is ask whether valley depth can decide how much context to preserve, not just where to cut, and whether that decision changes across languages. Three papers bear on this.
The first constrains the proposal. Is Semantic Chunking Worth the Computational Cost? found that threshold-based semantic chunking does not beat fixed-size chunking on natural English text. It won only on artificially stitched documents with abrupt "topic cliffs." We take this seriously. Our claim is therefore not that semantic boundaries are better in general. It is narrower: at matched compute, valley depth is a better signal for allocating overlap, and the advantage should be largest where fixed token counts are most misaligned with meaning.
The second paper motivates the language angle. The Token Tax shows that fertility, meaning tokens per word, explains 20–50% of accuracy variance across 16 African languages. Its argument is about question answering, not segmentation, and it implies nothing about where chunk seams land.
The third supplies the evaluation setting. MGAL is a long-context benchmark built on UN reports in the six official UN languages. It documents "local semantic crowding," where models lose discourse roles when adjacent sentences share topic and entities. It proposes no chunking method.
The closest method paper is Benchmarking and Improving Long-Text Translation with Large Language Models, whose Content Overlapping uses fixed token windows with uniform overlap across Zh/De/Ru–En. Our variable, depth-driven overlap is the direct contrast to it.
Research Program
The pipeline has three stages. Each stage can be swapped out independently, which the ablations exploit.
Scoring the text. The document is split into sentences, or into 20-word pseudosentences where punctuation is unreliable, following Hearst. Each unit is embedded with a multilingual sentence encoder. Similarity between adjacent blocks of $k$ units produces a curve, and each local minimum $j$ receives a depth score $d_j$ as in TextTiling. As a second scorer we use a mutual-information gap in the style of M³Seg. Because windows are counted in words or sentences, the curve should not depend on fertility. That is the first thing we test.
Placing seams. The downstream reader has a token budget $L$ per call. A dynamic program chooses seams to maximize total depth at the seams, subject to every chunk fitting within $L$. This is where fertility re-enters: a high-fertility language fits fewer words per chunk, so it has fewer valleys to choose from.
Allocating overlap. Given a total overlap budget $B$ in tokens, seam $j$ receives
$$o_j = B \cdot \frac{1 - \hat d_j}{\sum_i (1 - \hat d_i)},$$
where $\hat d_j \in [0,1]$ is the seam's depth normalized within the document. Deep valleys get almost no overlap. Seams forced into the middle of a topic get most of it. We normalize within each document, as Hearst's mean-minus-standard-deviation cutoff does, because absolute thresholds are known to be brittle across genres.
We also run a diagnostic. On parallel documents we measure the absolute deviation between each fixed-token seam and the nearest gold boundary, expressed in characters. We then correlate that deviation with the tokenizer's fertility for that language. A second, cleaner intervention re-tokenizes identical text with tokenizers of different vocabulary sizes. This varies fertility while holding language and content fixed.
Baselines
The naive reference is fixed-token chunking with no overlap, together with plain truncation. For truncation we use the middle-truncation strategy from LongBench, which keeps the instruction and the question and drops the centre of the document. Every other method must beat this reference to matter.
The strongest existing competitor is fixed-size chunking with overlap. Qu et al. found it matches or beats semantic chunking in English, reporting fixed-size F1@5 of 90.59% on HotpotQA and identical BERTScore of 0.76 against semantic chunking on DelucionQA. In translation, Content Overlapping, combined with mixed document lengths and multi-task training, moved d-BLEU on Novels from 18.1 to 19.2, against a GPT-4 average of 31.4 d-BLEU. We run fixed overlap at 50%, as recommended in InstructCoder, and at our matched budget, which defaults to 15% of document tokens. Every variant uses the same reader model, the same per-call budget $L$, the same encoder where one is needed, and the same small grid over window size $k$, tuned on a held-out English split only.
A fertility-normalized fixed baseline counts chunks in words or characters rather than tokens. This isolates the unit of measurement from the placement of seams.
The key ablation keeps our overlap schedule but shuffles depth scores across seams. It tests whether the benefit comes from depth-guided allocation or merely from the overall shape of the overlap distribution. A second ablation uses valley seams with zero overlap.
Beating the strongest baseline means one specific result. At a 15% budget, our method must match or exceed fixed 50% overlap on MGAL paragraph- and document-level accuracy and on d-BLEU. It must also clearly exceed fixed 15% overlap. Both must hold with confidence intervals from bootstrap resampling over documents.
For chunked pipelines on MGAL, no reported result was found on Lacuna. MGAL reports only a context ablation, 0.73 accuracy with context versus 0.31 without, which bounds the achievable range.
Why not just…?
Why not just count fixed chunks in words instead of tokens? This is the real threat. If the Token Tax harms chunking only because token budgets shrink the amount of text, then word-normalized fixed chunks remove the problem without any valley detection. That baseline is included precisely to test this. If it matches our method at matched compute across all six languages, we will report that Qu et al.'s negative result extends multilingually once units are normalized. The diagnostic study would then be the contribution. We expect it not to match fully for one reason: word normalization fixes chunk size but not where seams land, and the overlap budget is still spread uniformly.
Why not just feed the whole document to a long-context model? Longer windows do help. In MuLD, Longformer beat chunked T5 on character classification, 82.58 versus 54.01 F1. But LongBench reports a 17% drop for GPT-3.5-Turbo-16k beyond 8k tokens, and MGAL finds position bias and semantic crowding persisting at 128K. Cost also grows with the square of token count, so full context is most expensive exactly for high-fertility languages. We include full-context reading as an upper reference wherever the document fits, and report accuracy per unit of compute.
Why not just split at sentences and pack them greedily? Sentence-aligned packing avoids cutting clauses in half and is cheap, so we include it. It does not know where topics shift, and it offers no rule for allocating overlap. On ASR-style text without punctuation it depends on a splitter. Cross-lingual and cross-domain discourse segmentation of entire documents found a UDPipe splitter trailing learned segmentation on German, 69.7 versus 76.6 F1.
Why not just train a supervised segmenter? The supervised cross-lingual segmenter in that same paper works at the level of individual clauses and needs labelled data. Most target languages lack topic-level labels, so an unsupervised valley scorer is the practical option. A supervised scorer could be dropped into the scoring stage if labels become available.
Proposed Experiments and Decisive Checks
Detector check. On MGAL's UN reports, we strip the section headings and use them as gold topic boundaries. We then measure $P_k$ and WindowDiff for token-unit versus word-unit windows across all six languages, and under the tokenizer-swap intervention. The prediction is that token-unit error rises with fertility while word-unit error stays flat. We also report English AMI and ICSI to confirm that our scorer is near M³Seg's 0.248 and 0.203.
Downstream check. We run a chunk-then-aggregate reader on three tasks:
- MGAL paragraph- and document-level tasks;
- Wang et al.'s Novels-XL and Subtitle Ru–En translation, scored with d-BLEU plus a character-level metric, to limit the tokenization distortion in surface metrics;
- LongBench's Chinese and English subsets.
The decisive result is a curve for each language of performance against total tokens processed. Our method should dominate fixed overlap across the curve. Its margin over the word-normalized baseline should grow with fertility. It should beat the shuffled-depth ablation.
Controls. We use stitched documents in the style of Qu et al. as a positive control. A null result on natural text but a win on stitched text would reproduce their finding rather than refute our method, and we would report it as such.
Limitation. The fertility range covered by UN languages is modest, so the tokenizer-swap intervention stands in for the high-fertility regime the Token Tax studied. Extending the test to genuinely low-resource long documents remains future work.
References
- Multi-Paragraph Segmentation of Expository Text
- A texttiling based approach to topic boundary detection in meetings
- M³Seg: A Maximum-Minimum Mutual Information Paradigm for Unsupervised Topic Segmentation in ASR Transcripts
- Is Semantic Chunking Worth the Computational Cost?
- The Token Tax: Systematic Bias in Multilingual Tokenization
- MGAL: A Multilingual Granularity-Aware Long-Context Benchmark
- Benchmarking and Improving Long-Text Translation with Large Language Models
- LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding
- InstructCoder: Instruction Tuning Large Language Models for Code Editing
- MuLD: The Multitask Long Document Benchmark
- Cross-lingual and cross-domain discourse segmentation of entire documents