Word-Level Alignment of Paper Documents with their Electronic Full-Text Counterparts

Mark-Christoph Müller, Sucheta Ghosh, Ulrike Wittig, Maja ReyNAACL 2021

TL;DR

This paper presents an unsupervised method to automatically align words between scanned paper documents and their electronic full-text versions. This is crucial for bridging the "paper-to-digital" gap in scientific workflows like manual data curation and improving tools for biomedical expression OCR.

What Problem Does This Paper Address?

In many scientific fields, particularly life sciences, researchers still rely on paper printouts for tasks like close reading and manual data annotation, even though the original literature is digital. This creates a "media break" where information marked on paper needs to be manually re-entered into digital databases. Furthermore, existing OCR technology struggles with complex scientific expressions, hindering the creation of high-quality training data for specialized OCR systems. This paper tackles the challenge of automatically creating a word-level link between the physical paper document and its digital counterpart, thereby facilitating data curation and enabling the generation of ground-truth data for advanced OCR.

Methods

The core of the approach involves converting both the scanned paper document and its electronic full-text counterpart into a common, multilevel XML format (MMAX2).

  1. OCR and XML Conversion: Scanned document images are processed using the Tesseract OCR engine to generate semi-structured text with character-level bounding boxes. This output, along with the electronic full-text (obtained in .nxml format from sources like PubMedCentral), is then converted into the MMAX2 XML format. This shared format allows for capturing content and structural information, even if tokenization differs.
  2. Sequence Alignment: A dynamic programming algorithm, specifically the Needleman-Wunsch algorithm (implemented via Biopython), is used to perform a word-level alignment between the token lists from the OCR'd document and the electronic full-text.
  3. Pre- and Post-processing: To improve alignment accuracy, several optional steps are employed:
  4. Pre-compression: Identical token sequences across both documents are compressed to reduce computational complexity.
  5. De-hyphenation: Hyphenated words in the OCR output that are not hyphenated in the electronic version are reconstructed.
  6. Joining/Splitting Tokens: Incorrectly split or joined tokens in the OCR output are corrected based on the electronic version.
  7. Force-align: A post-processing step to manually align tokens that the primary algorithm missed, particularly when character sequences and context match.

Datasets

The authors used a dataset of 68 documents that were freely available as both full-text XML and PDF. They created two types of image versions for each document:

  • DOIconv: High-resolution PNG images generated directly from PDFs.
  • DOIscan: Images extracted from "sandwich" PDFs, which contain both the document text (from the provider's OCR) and a background image.

Key Insights and Ideas

  • Bridging the Media Break: The paper demonstrates a practical, unsupervised method to overcome the manual effort required to link information from paper documents to their digital counterparts, a significant bottleneck in scientific curation.
  • Unsupervised Approach: The method does not require manual annotation or training data, making it broadly applicable.
  • OCR Quality vs. Conversion: The study found that the quality difference between directly converted PDF images (DOIconv) and scanned images (DOIscan) had a surprisingly small impact on alignment performance, suggesting that converted PDFs can be functionally equivalent for tasks like OCR ground-truth generation.
  • Highlighting Detection: The method can also detect and transfer manual highlighting from scanned documents, which is valuable for visualizing important sections in the aligned electronic version.

Key Results

The alignment procedure achieved high precision (around 95%) and recall, with the best F-scores reaching 86.63% for converted PDF documents (DOIconv) and 85.20% for scanned documents (DOIscan) when all pre- and post-processing steps were applied. The authors note that the inferior quality of scanned images resulted in slightly lower scores but the absolute difference was minimal.

Limitations and Future Work

  • Limitations: The highlighting detection mechanism relies on "inpainted" OCR background images, which are not produced by all OCR engines (like Tesseract by default). While alternative software exists, this could be a constraint. The conversion of embedded LaTeX math code is limited to simpler expressions.
  • Future Work: The authors plan to implement solutions for the identified use cases (biocuration, document triage, biomedical expression OCR) and to create OCR ground-truth data at a larger scale. They envision their work contributing to more natural and efficient scientific workflows.

Summary updated

BibTeX
@inproceedings{muller2021word,
  author = {Mark-Christoph Müller and Sucheta Ghosh and Ulrike Wittig and Maja Rey},
  title = {{Word-Level Alignment of Paper Documents with their Electronic Full-Text Counterparts}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2021},
}

Loading related content...

Figure