Do Restoration Models Rewrite Ancient Archives Toward Their Most Copied Texts?

Machine learning is increasingly used to fill the broken parts of damaged ancient texts: a model reads the surviving words around a gap on a cuneiform tablet and guesses what was written there. These systems are scored on whether they recover the right word. Nobody has asked which source the guess quietly copies from. Many Babylonian literary and scholarly texts survive in several manuscripts that disagree in small ways. A model trained on the whole corpus may fill each gap with the reading from the most widely copied version, overwriting the local or minority variant that the damaged tablet actually carried.

If that happens at scale, computational restoration homogenizes the record: it makes the surviving archive look more uniform than it was. The same flattening has been measured in modern scholarship. When large language models suggest references, they favor already famous papers, and their suggestions have a median citation count 1,326 higher than the true references (Large Language Models Reflect Human Citation Patterns with a Heightened Citation Bias). This note proposes to measure that "rich-get-richer" bias in restoration of fragmentary texts, using a citation-style map of which manuscripts share text with which. It then tests whether a restoration model that draws on the damaged tablet's own textual lineage, rather than the whole corpus, removes the bias without losing accuracy.

Core Claim

Lacuna restoration in cuneiform has a structural popularity bias that per-token accuracy cannot see. Where manuscripts of the same composition disagree, models choose the reading attested in more manuscripts far more often than the ground truth warrants. This bias is larger than what word frequency alone predicts. Restoration conditioned on the fragment's own neighborhood in a manuscript-parallel graph should cut the bias and raise accuracy on minority-variant gaps.

The new contribution has two parts. The first is a measurement: an "attestation gap" for restorations, modeled on the citation gap in reference prediction. The second is a test of whether lineage-local retrieval beats global retrieval and plain fine-tuning on the gaps that carry local textual identity.

Three findings motivate the claim. The EvaCun organizers report that token prediction in broken tablets reaches only 21% for the best shared-task system and roughly 0% on out-of-vocabulary tokens (EvaCun 2025 Shared Task). Jon and Bojar observed that their fine-tuned language models over-predict common words and under-predict rare ones (Finetuning LLMs for EvaCun 2025 token prediction shared task). Restoration of Joseon Hanja records reaches 89.09% top-10 accuracy overall but 8.3% on proper nouns (Restoring and Mining the Records of the Joseon Dynasty). Together these suggest that models succeed where text is standardized and fail on what is local.

One result constrains the claim. LSTMs reach 85% top-1 accuracy on formulaic Late Babylonian administrative texts (Restoration of Fragmentary Babylonian Texts Using Recurrent Neural Networks). The bias should therefore matter little for formulaic genres, and the study must show that difference rather than assume homogenization everywhere.

Research Program

The first step is building the manuscript-parallel graph, which plays the role a citation graph plays in bibliometrics. The data come from the Electronic Babylonian Library: about 25,000 fragments and over 350,000 transliterated lines, with 1,250 identified joins, where a join is a match of physically separated pieces back to one tablet (Transliterated Cuneiform Tablets of the Electronic Babylonian Library Platform). We align lines across fragments using two signals: lemma sequences produced by a lemmatizer such as BabyLemmatizer, which reaches 93% in EvaCun, and edit distance on the transliterations.

Two fragments are linked when they share aligned lines. Each node gets an attestation count, meaning how many manuscripts carry its reading of each shared line. Community detection then groups fragments into lineages. Lineages are a proxy for recension, provenance or scribal school, and we check the proxy against eBL composition and provenance metadata.

This differs from intertextual tools such as Latin BERT allusion search (Augmented Close Reading for Classical Latin). That tool helps a reader find allusions, with over 80% recall on attested ones. We use parallels to audit a generator, and we work at the level of the network rather than single passages.

The second step is a variant-sensitive test set. We mask intact spans in held-out fragments at positions where aligned manuscripts disagree, so the true reading is known from the tablet itself. Each masked position is labeled by whether the true reading is the majority variant or a minority variant within its aligned group. A frequency-matched control set uses positions with no attested variation, where the true token has the same corpus frequency.

The third step is attribution and three metrics. Each restoration is attributed to the manuscripts whose attested line contains the predicted n-gram. The variant-preservation rate is the share of minority-variant gaps restored to the minority reading. The attestation gap is the median attestation count of the attributed source minus that of the true source; this mirrors the citation gap. Finally, a Gini coefficient compares how concentrated the attributed sources are against how concentrated the true sources are.

The fourth step is the intervention, lineage-local restoration. A fine-tuned model receives, in its prompt, aligned parallel lines retrieved only from the damaged fragment's own graph community. Minority readings from that community are kept even when they are outvoted corpus-wide.

Baselines

The naive reference point is majority-parallel copying. For each gap, it inserts the reading attested in the most aligned manuscripts, and falls back to the most frequent token when no parallel exists. It is the maximally homogenizing restorer. Any model that does not beat it on minority-variant gaps is adding no lineage information. It runs on exactly the same graph and test set.

The strongest existing method is the QLoRA-fine-tuned model ensemble of Finetuning LLMs for EvaCun 2025 token prediction shared task. It fine-tunes Aya Expanse 8B, Command-R 34B and Mistral Small 3 24B with low-rank adapters and combines their outputs by majority vote across prompts. The paper reports 22.1% single-model and 26.9% ensemble development accuracy; the shared-task best was 21% (Team 32). We reproduce it on the authors' training split of 913,252 tokens, use the same base models, adapters and sampling temperature of 0.2, and score it both on the official 1,500 held-out examples and on our variant set. Majority voting may itself sharpen popularity bias, which makes this baseline an essential comparison rather than a straw man.

The architecture-matched baseline is a bidirectional LSTM re-ranker in the style of Restoration of Fragmentary Babylonian Texts Using Recurrent Neural Networks. It is retrained on eBL to check that the bias is not specific to large language models.

The ablation that isolates the key ingredient is global-retrieval restoration. It is identical to the proposal, with the same model, prompt length and number of retrieved lines, except that parallels come from the whole corpus, ranked by similarity. If global retrieval matches lineage-local retrieval on variant preservation, then the graph communities contribute nothing.

All systems receive the same tuning budget, measured in GPU-hours, and the same number of hyperparameter trials. Counting as beating the strongest baseline requires two things on minority-variant gaps, with bootstrap confidence intervals excluding zero. The first is a higher variant-preservation rate together with a smaller attestation gap. The second is no more than one point lower accuracy on the official EvaCun evaluation.

Why not just…?

Why not just measure word-frequency calibration instead of building a graph? Jon and Bojar already show the models over-predict common words, so the homogenization may be nothing more than word frequency. The frequency-matched control settles this. If the variant-preservation deficit on minority-variant gaps is no larger than the error on frequency-matched control positions, the graph framing adds nothing, and we will report that outcome. We expect a larger deficit, because minority variants are often common words used in an unusual position or order.

Why not just raise the sampling temperature or sample more diverse outputs? Sampling adds randomness that does not depend on the fragment, so it should spread errors around rather than recover the right minority reading. Its effect on accuracy is also a real risk. We sweep temperature for the baseline and plot variant preservation against overall accuracy. If temperature reaches the same point on that curve as lineage-local retrieval, the method is unnecessary. This is a genuine threat, and the curve decides it.

Why not just use retrieval of parallels, which philologists already do by hand? Global retrieval is exactly the ablation. The hypothesis is that unrestricted retrieval imports the majority reading and amplifies homogenization, the way models that suggest references gravitate to famous papers (Large Language Models Reflect Human Citation Patterns with a Heightened Citation Bias). If global retrieval preserves variants as well as lineage-local retrieval, we will conclude that retrieval alone suffices.

Proposed Experiments

The first experiment is the measurement study. All baselines are scored on the variant set, stratified by genre. The decisive check is that minority-variant preservation falls well below the rate that the true majority/minority mix in the test set would produce for an unbiased restorer. The bias should be negligible in formulaic administrative texts, which can be checked against an Achemenet-style genre split, and pronounced in literary and scholarly compositions. If the bias is uniform across genres, or absent, the core claim fails.

The second experiment is the intervention. We compare lineage-local restoration with global retrieval and with the ensemble on variant preservation, attestation gap, the Gini of attributed sources, and official EvaCun accuracy.

The third experiment tests the graph itself. We perturb the community assignments by shuffling them within the same genre, and check whether the gains vanish. They should vanish if the graph's lineage structure, rather than simply retrieving more context, is what helps.

A secondary replication targets the Joseon records. Their very low proper-noun accuracy suggests the same kind of local erasure. There we would map duplicated passages between the Annals and the Royal Secretariat diaries in the same way, and report whether the attestation gap appears in a second writing system. We report that study only if the cuneiform result holds.