A printed or hand-copied historical page is highly repetitive. The same letter, ligature and stroke shape recurs hundreds of times on a single folio, written by the same scribe or cut from the same type. When one instance is faded, stained or torn, its intact siblings elsewhere on the page are the most faithful evidence of what it looked like. Classical restoration tools exploit this kind of repetition by patch matching: find other small image squares that look similar, then average them. The difficulty is that this search breaks down exactly when it is most needed. Once the corruption is stronger than the differences between shapes, the similarity score measures coincidences in the damage rather than in the letters. The search then returns "false friends", and averaging over them destroys strokes.
On historical pages the false friends are especially dangerous, because the most common heavy corruption is ink bleed-through, where text from the back of the sheet shows through mirrored. That corruption is itself text-like. We propose to measure where glyph matching breaks down on compound document degradation, and to build a matcher that holds up past that point. The matcher uses a descriptor trained to ignore the page's specific damage. It also applies an explicit per-match test that refuses to borrow evidence when the damage makes the correspondence unidentifiable. The restorer then copies stroke evidence only from matches that pass the test and records, for every pixel, which intact glyphs it came from. This gives a restoration anchored to the scribe's own hand rather than to a language model's guess.
Core Claim
The claim has two parts, and each can be tested on its own. The first is empirical. On historical pages with compound, spatially uneven damage, every existing way of scoring patch similarity has a severity threshold beyond which the precision of its top matches collapses. We expect that threshold to arrive much earlier for bleed-through than for additive noise of equal energy, because bleed-through violates the zero-mean assumption that noise-cancelling features rely on. The second is methodological. A descriptor trained contrastively on same-glyph pairs under independent compound damage, with mirrored verso text as a hard negative, combined with a match-rejection rule derived from the breakdown condition, moves that threshold substantially. Restoring only from accepted matches then beats both large end-to-end restorers and language-prior inpainting on damaged glyphs, and it hallucinates less.
The breakdown condition is standard. For observed patches $Y_A = A + N_A$ and $Y_B = B + N_B$, with clean content $A, B$ and degradations $N_A, N_B$, the expected squared distance under independent zero-mean degradation is
$$\mathbb{E}\|Y_A - Y_B\|^2 = \|A - B\|^2 + \mathrm{tr}(\Sigma_A + \Sigma_B),$$
where $\Sigma_A, \Sigma_B$ are the degradation covariances. This decomposition is the one Feature Extraction for Patch Matching in Patch-based Denoising Methods uses to explain BM3D's collapse at very high noise. That paper's remedy averages pixels over half-patch regions so that zero-mean noise cancels. The remedy is sound for Gaussian noise, but it assumes the corruption has mean zero. Bleed-through, oxidation gradients and stains are not zero-mean. They add a structured term to $\|A-B\|^2$ itself.
Our gate works in a learned feature space $f(\cdot)$. Let $d_1$ and $d_2$ be the distances from a query patch to its best and second-best candidate glyph clusters, and let $\hat\sigma$ be a predicted spread of residual degradation in feature space for the two patches involved. We accept a match only when
$$d_2 - d_1 > \kappa\,\hat\sigma,$$
with $\kappa$ calibrated on held-out synthetic pages to a target match precision. In plain terms, borrowing is allowed only when the winning candidate beats the runner-up by more than the damage could plausibly explain. Because $\hat\sigma$ is predicted per patch, the rule adapts to damage that varies across the page. This is the property DocPure says its single global routing step lacks.
Research Program
The work has three parts, and the first is a measurement study that stands on its own even if the method disappoints.
Measuring breakdown curves. We render text in several scripts and hands, including Latin print with long s, Fraktur-style type, and Sharada-like cursive. Where real text lines are available we use them, and we place them on real paper backgrounds. We then apply compound damage with controlled, spatially varying severity: mirrored and blurred verso text at varying opacity, fading, additive and impulse noise, stain fields, and small non-rigid warps. We know the identity of every glyph, so we can plot top-$k$ match precision (the fraction of retrieved patches showing the same character) against severity for each similarity measure. The measures are raw Euclidean distance as in BM3D, Chen and Krzyżak's rotating regional means, the mixed-resolution vectors of Mixed-Resolution Patch-Matching, which keep fine detail that coarse pyramid levels would drop and which matters for thin strokes, and the learned affinities of Non-Local Recurrent Network for Image Restoration. That paper embeds non-local matching in a recurrent network and finds that restricting search to a 45-pixel neighborhood suppresses false friends on natural images. Page-scale glyph repetition sits far outside such a window, so its finding constrains our design. We cannot simply search globally with ungated scores. We then check the synthetic curves against real pages that carry character-level labels, namely FPHDR and the Sharada collections used by CURIO.
The matcher. The descriptor is a small convolutional encoder on stroke-scale patches, sized by an estimated stroke width in the spirit of the Stroke Width Transform. Xiong et al. use that transform to scale morphology to the pen width, and we borrow the scale estimate as a component. The encoder is trained contrastively. Positives are the same glyph instance under two independent draws of compound damage. Hard negatives are other glyphs, mirrored verso glyphs at matched contrast, and stain edges. A second head predicts $\hat\sigma$. We search the whole page, or the whole folio set from the same hand, using approximate nearest-neighbor indexing over glyph-scale patches, so memory stays linear in page size.
The restorer. For each damaged region, accepted matches are aligned with a small learned warp, because inter-glyph variation in handwriting is non-rigid. Their ink layers are aggregated by robust median-style fusion, and the result is passed together with the observed pixels to a lightweight local restorer. Where no match passes the gate, the restorer runs alone and the pixel is marked "unsupported". The output is a restored page plus a provenance map. This contrasts with AutoHDR, which locates damage with OCR confidence, predicts missing characters with a language model, and redraws them with a patch-autoregressive diffusion model. We never create a glyph from a linguistic prior. We copy the page's own instances, or we abstain. The contrast with LayeredDoc is different. It separates text from stamps into layers to keep forensic integrity but cannot recover destroyed strokes, while we can recover them when evidence exists elsewhere on the page.
Baselines
The naive reference is the unrestored page fed to the same downstream readers, together with standard BM3D-style Euclidean patch matching used as preprocessing. The file reports its collapse on natural images: 18.45 dB versus 21.41 dB for regional-mean matching on Lena at $\sigma=240$, per Chen and Krzyżak. It also reports raw OCR on severely damaged FPHDR pages at 46.83% accuracy, per AutoHDR.
The strongest competitor on damaged-glyph recovery is AutoHDR, which reports that its restoration lifts OCR accuracy on FPHDR from 46.83% to 84.05%. We will run the released system if it is available, and otherwise report its published number on the same split. Beating it means matching or exceeding 84.05% OCR accuracy without human review, while producing fewer glyphs judged unsupported by the page's evidence and running far faster than its reported five minutes per page on an A10.
The strongest general document restorers are DocRes and DocPure. DocRes is a 15.2M-parameter Restormer conditioned on handcrafted hint maps such as Sauvola thresholds and background estimates. DocPure reports 32.55 dB PSNR on the Text Deblurring Dataset. We will retrain both on our compound synthetic data with the same number of training images and GPU-hours as our model, and with an equal hyperparameter sweep budget. No standalone DocRes DIBCO number was found on Lacuna.
For binarization the classical bar is Xiong et al., with 94.26% F-measure on H-DIBCO 2012. The learned bar is SauvolaNet, a differentiable multi-window Sauvola layer with 40k parameters that reports 97.83% F-measure on H-DIBCO 2014. Because those benchmarks are close to ceiling and only moderately degraded, we do not expect large gains there. Our success criterion is no loss on standard DIBCO together with clear gains on our severe-damage splits.
Three ablations isolate the key ingredients. The first keeps our pipeline but replaces the learned descriptor with raw pixels or with regional means, which isolates the descriptor. The second keeps the descriptor but removes the gate and borrows from the top-$k$ matches unconditionally, which isolates the breakdown test. The third restricts search to a 45-pixel window as NLRN does, which tests whether page-scale search is actually needed.
Why not just…?
Why not just train Restormer or NAFNet end-to-end on synthetic compound damage? This is the most serious threat. The AutoHDR authors report that NAFNet and Restormer produce blotchy strokes and inconsistent style on severely eroded pages. DocPure reports ghosting and over-restoration when multi-degradation models lack structural routing. Neither result establishes that more capacity or global attention cannot close the gap. We will therefore train a Restormer variant with page-level global attention at matched compute on identical data. If it matches our method on glyph recovery under full local erasure, where local context cannot help and only distant instances carry information, then the explicit matcher is unnecessary and we will say so. Our prediction is that it will not, because its attention is exactly an ungated similarity score and should inherit the breakdown curve we measure.
Why not just use SWT, Sauvola, or MRF graph cuts? These methods are fast and need no training. The reason they fall short here is locality. Xiong et al. note that their method fails when edge gradients are obliterated. Wolf's dual hidden MRF separates recto from verso with first-order local neighborhoods under a stationary model, and Rowley-Brooke et al. need aligned two-sided scans and borrow exemplars only for paper background. None of them can recover a stroke that has been erased. We keep these methods as baselines and reuse stroke-width estimation, but the erased-glyph split will show whether local methods cap out.
Why not skip restoration and read the page with a vision-language model? CHURRO, a fine-tuned 3B Qwen 2.5 VL, reports 82.3% and 70.1% normalized Levenshtein similarity on printed and handwritten text. Its authors also document hallucinations from zero-shot models on damaged lines (CHURRO). This alternative is a real threat for transcription as the end goal, so we test it directly by running CHURRO on raw and on restored pages. If restoration does not raise its accuracy, the value of our method reduces to visual and forensic output plus reduced hallucination, which we measure separately.
Proposed Experiments
The decisive experiment is the breakdown curve. For each similarity measure we plot top-5 same-glyph precision against damage severity, separately for additive noise and for bleed-through of equal energy. The hypothesis fails if the learned descriptor without the gate already holds up as well as the gated version, or if the bleed-through curve does not drop earlier than the noise curve for the zero-mean-based features.
The second experiment is glyph recovery under controlled erasure. On synthetic pages, and on real pages with artificially masked characters, we erase 30–100% of a glyph's strokes. We then measure character error rate from a fixed recognizer and a hallucination rate, defined as the fraction of output characters that are confidently rendered but wrong, and we report both at the gate's operating points. A trustworthy method should trace a clean trade-off between how often it abstains and how accurate it is when it does answer. AutoHDR's design cannot abstain.
The third experiment is real-archive evaluation: OCR accuracy on FPHDR against AutoHDR's 84.05%, CER on the Sharada UUV set against CURIO's 6.9%, and F-measure on H-DIBCO 2012/2014 to confirm we give nothing up on standard binarization.
The fourth experiment targets spatially uneven damage. We build test pages that combine a corner stain, central bleed-through and edge blur, and compare single-pass restoration against DocPure run in its required multiple passes.
Beyond these accuracy numbers, we will report wall-clock time per page and peak memory, and we will check that the provenance maps mark every recovered glyph with a source instance that human raters judge to be the same letter.
References
- Feature Extraction for Patch Matching in Patch-based Denoising Methods
- DocPure: Prompt-Free Unified Document Restoration via Degradation-Aware Structure-Guided Wavelet Modulation
- Mixed-Resolution Patch-Matching
- Non-Local Recurrent Network for Image Restoration
- CURIO: Curvature-Aligned and Efficient OCR for Low-Resource Historical Manuscripts
- An enhanced binarization framework for degraded historical document images
- Reviving Cultural Heritage: A Novel Approach for Comprehensive Historical Document Restoration
- LayeredDoc: Domain Adaptive Document Restoration with a Layer Separation Approach
- DocRes: A Generalist Model Toward Unifying Document Image Restoration Tasks
- SauvolaNet: Learning Adaptive Sauvola Network for Degraded Document Binarization
- Document Ink Bleed-Through Removal with Two Hidden Markov Random Fields and a Single Observation Field
- A Non-parametric Framework for Document Bleed-through Removal
- CHURRO: Making History Readable with an Open-Weight Large Vision-Language Model for High-Accuracy, Low-Cost Historical Text Recognition