Transforming scholarship in the archives through handwritten text recognition
Günter Mühlberger, Louise Seaward, Melissa Terras, Sofia Ares Oliveira, Vicente Bosch, Maximilian Bryan, Sebastian Colutto, Hervé Déjean, Markus Diem, Stefan Fiel, Basilis Gatos, Albert Greinoecker, Tobias Grüning, Günter Hackl, Vili Haukkovaara, Gerhard Heyer, Lauri Hirvonen, Tobias Hodel, Matti Jokinen, Philip Kahle, Mario Kallio, Frédéric Kaplan, Florian Kleber, Roger Labahn, Eva Maria Lang, Sören Laube, Gundram Leifert, Georgios Louloudis, Rory McNicholl, Jean-Luc Meunier, Johannes Michael, Elena Mühlbauer, Nathanael Philipp, Ioannis Pratikakis, Joan Puigcerver Pérez, Hannelore Putz, George Retsinas, Verónica Romero, Robert Sablatnig, Joan-Andreu Sánchez, Philip Schofield, Giorgos Sfikas, Christian Sieber, Nikolaos Stamatopoulos, Tobias Strauß, Tamara Terbul, Alejandro H. Toselli, Berthold Ulreich, Mauricio Villegas, Enrique Vidal, Johanna Walcher, Max Weidemann, Herbert Wurster, Konstantinos Zagoris
For decades, archives have been digitizing their collections, creating vast repositories of images. However, these images remain functionally "silent." While Optical Character Recognition (OCR) has mastered printed text, it fails against the variability of human handwriting. Until recently, the only way to "read" these digital archives was through painstaking manual transcription—a process that does not scale. This paper introduces Transkribus, a platform developed under the EU H2020 READ project, which aims to bridge this gap by transforming "dark data"—millions of unsearchable digitized manuscript images—into searchable, machine-readable text.
The Transkribus Ecosystem
Transkribus is not just a tool but a comprehensive infrastructure designed to facilitate collaboration between humanities scholars, computer scientists, and the public. The platform provides a user-friendly interface that allows non-technical researchers to train their own AI models, effectively democratizing access to deep learning-powered Handwritten Text Recognition (HTR).
The Transkribus ecosystem connects archives, scholars, technology providers, and the public.
The core of Transkribus is an HTR engine based on Deep Neural Networks (DNNs). Unlike traditional systems that attempt character-by-character recognition, Transkribus employs a line-oriented approach.
Layout Analysis and Ground Truth
The process begins with layout analysis, where the system identifies "baselines"—horizontal lines running underneath text. This segmentation is critical for the HTR engine to extract text line images. Following segmentation, users must create "Ground Truth" by manually transcribing a subset of pages (typically around 15,000 words or 75 pages). This data is used to train a model specific to a particular hand or script style.
The Transkribus interface allows users to manually transcribe lines to create ground truth for training.
The platform utilizes engines like CITlab or the TensorFlow-based HTR+, which learn visual patterns associated with the provided text. HTR+ represents a significant advancement, offering up to 10 times faster training and a 5-10% improvement in accuracy over previous iterations. For documents where accuracy is insufficient for full transcription, the system provides Keyword Spotting (KWS). KWS uses confidence matrices to allow users to search for specific terms even in "messy" results with high error rates (up to 30% CER), acting as a vital safety net for researchers.
Performance and Impact
The platform's efficacy has been validated across diverse archival datasets, including the Bentham Papers (UCL), German death registers, and Finnish WWII diaries. Key results include:
High Accuracy: Models for the Bentham and Greifswald collections achieved a 95% character accuracy (5% Character Error Rate).
Scalability: Automated transcription can process 50,000 pages in approximately 33 days, a task that would take human transcribers years.
The 5% Threshold: Research identifies a "sweet spot" in Character Error Rate. When CER falls below 5%, automated transcripts become highly reliable and faster to proofread than manual transcription.
Language Independence: Because neural networks learn visual patterns rather than linguistic rules, the system is inherently language-independent, successfully processing everything from medieval Hebrew to 20th-century Cyrillic.
Challenges and Sustainability
Despite its success, challenges remain in handling complex layouts, such as tabular data or overlapping marginalia. Furthermore, if the CER exceeds 10%, the time required for manual correction can outweigh the benefits of automation. To ensure long-term sustainability beyond grant funding, the READ-COOP was established as a cooperative model for institutions and users to share infrastructure costs. Future work focuses on "out-of-the-box" models requiring zero user training and improved structural recognition for complex document elements.
Summary updated
Abstract
An overview of the current use of handwritten text recognition (HTR) on archival manuscript material, as provided by the EU H2020 funded Transkribus platform. It explains HTR, demonstrates Transkribus, gives examples of use cases, highlights the affect HTR may have on scholarship, and evidences this turning point of the advanced use of digitised heritage content. The paper aims to discuss these issues.,This paper adopts a case study approach, using the development and delivery of the one openly available HTR platform for manuscript material.,Transkribus has demonstrated that HTR is now a useable technology that can be employed in conjunction with mass digitisation to generate accurate transcripts of archival material. Use cases are demonstrated, and a cooperative model is suggested as a way to ensure sustainability and scaling of the platform. However, funding and resourcing issues are identified.,The paper presents results from projects: further user studies could be undertaken involving interviews, surveys, etc.,Only HTR provided via Transkribus is covered: however, this is the only publicly available platform for HTR on individual collections of historical documents at time of writing and it represents the current state-of-the-art in this field.,The increased access to information contained within historical texts has the potential to be transformational for both institutions and individuals.,This is the first published overview of how HTR is used by a wide archival studies community, reporting and showcasing current application of handwriting technology in the cultural heritage sector.
BibTeX
@article{muhlberger2019transforming,
author = {Günter Mühlberger and Louise Seaward and Melissa Terras and Sofia Ares Oliveira and Vicente Bosch and Maximilian Bryan and Sebastian Colutto and Hervé Déjean and Markus Diem and Stefan Fiel and Basilis Gatos and Albert Greinoecker and Tobias Grüning and Günter Hackl and Vili Haukkovaara and Gerhard Heyer and Lauri Hirvonen and Tobias Hodel and Matti Jokinen and Philip Kahle and Mario Kallio and Frédéric Kaplan and Florian Kleber and Roger Labahn and Eva Maria Lang and Sören Laube and Gundram Leifert and Georgios Louloudis and Rory McNicholl and Jean-Luc Meunier and Johannes Michael and Elena Mühlbauer and Nathanael Philipp and Ioannis Pratikakis and Joan Puigcerver Pérez and Hannelore Putz and George Retsinas and Verónica Romero and Robert Sablatnig and Joan-Andreu Sánchez and Philip Schofield and Giorgos Sfikas and Christian Sieber and Nikolaos Stamatopoulos and Tobias Strauß and Tamara Terbul and Alejandro H. Toselli and Berthold Ulreich and Mauricio Villegas and Enrique Vidal and Johanna Walcher and Max Weidemann and Herbert Wurster and Konstantinos Zagoris},
title = {{Transforming scholarship in the archives through handwritten text recognition}},
journal = {J. Documentation},
year = {2019},
doi = {10.1108/jd-07-2018-0114},
}