Textual and Visual Characteristics of Mathematical Expressions in Scholar Documents

Vidas DaudaraviciusNAACL 2019

TL;DR

This paper introduces a large, open-access corpus of scholarly documents with parallel PDF and XML formats, enabling the analysis of mathematical expressions (MEs). The authors reveal distinct textual and visual characteristics of inline vs. display MEs, offering valuable insights for improving ME recognition and translation tasks, particularly for image-to-LaTeX conversion.

What Problem Does This Paper Address?

Mathematical expressions (MEs) are fundamental to scientific communication, yet their representation and analysis in digital documents pose significant challenges. While LaTeX is a common markup for MEs, its non-linear, visual nature makes it difficult for machines to parse and understand. Existing methods for extracting MEs from documents, especially PDFs, often struggle with the complex interplay of text and visual layout. Furthermore, there's a lack of comprehensive understanding of the inherent textual and visual properties of MEs across different contexts (inline vs. display), which hinders the development of robust automated tools for tasks like information retrieval, machine reading, and translation.

Methods

The core of this work is the creation and analysis of a novel corpus. The authors collected a large set of open-access scholarly documents that are available in both PDF and JATS-XML formats, ensuring parallel content. They then developed methods to extract LaTeX-encoded mathematical expressions from these documents. The analysis focuses on characterizing these expressions based on:

  • Textual Length: Measured in the number of LaTeX tokens.
  • Visual Size: Analyzed through rendered image dimensions (width and height in pixels).
  • Tokenization: Investigating how whitespace characters affect ME representation and proposing alternative tokenization strategies.

They differentiate between "inline" MEs (embedded within regular text) and "display" MEs (presented on separate lines), recognizing that their visual and textual characteristics differ.

Datasets

The paper introduces a new dataset:

  • OAScholarXML Corpus: This corpus consists of 8599 open-access scholarly documents. Each document is available in both PDF and JATS-NISO-XML formats, and crucially, they all contain at least one mathematical expression. The documents were published between 2012 and 2018 and are under permissive CC licenses. The corpus contains over 1.2 million distinct annotated formulae and more than 80 million raw LaTeX tokens.

Key Insights and Ideas

  • Distinct Inline vs. Display ME Characteristics: Inline MEs are generally shorter and have more constrained visual dimensions compared to display MEs, which can be significantly longer and wider. This distinction is crucial for developing specialized processing pipelines.
  • Vocabulary Saturation: The vocabulary of LaTeX tokens used in mathematical expressions appears to be relatively small and has reached saturation. This suggests that a fixed, albeit large, set of tokens can cover most MEs.
  • Whitespace Caution: Whitespace characters in LaTeX MEs do not always correspond to visual separation and can be misleading for tokenization. They should be treated as potential tokens themselves.
  • Image Size Bounds: The paper provides concrete pixel ranges for inline and display ME images, which are valuable for ME detection in PDFs. These ranges are more extensive than those previously used in some image-to-LaTeX translation tasks.
  • Display ME Decomposition: The authors propose that complex display MEs, due to their length and visual complexity, might be best handled by decomposing them into arrays of smaller, single-line display images for easier processing.

Key Results

  • Corpus Size: The OAScholarXML corpus contains over 1.2 million distinct MEs and over 80 million raw LaTeX tokens across more than 8,000 documents.
  • ME Frequency: The average document contains 291 MEs, with an average of 1936 distinct MEs. Approximately 60% of MEs are duplicated within a document.
  • Inline ME Length: The peak length for distinct inline MEs is around 10 tokens, with most falling below 75 tokens.
  • Display ME Length: Display MEs can be significantly longer, with the longest observed being 13987 tokens, and many exceeding 500 tokens.
  • Image Dimensions:
    • Inline MEs: Image widths range from 30-500 pixels, and heights from 20-40 pixels.
  • Display MEs: Image widths range from 100-1000 pixels, and heights from 20-300 pixels (with partial display image heights ranging from 20-85 pixels).

Limitations and Future Work

  • Strict Bounds: The paper questions the strict, unexplained bounds used in some prior image-to-LaTeX translation work and suggests their analysis provides a more realistic view.
  • Future Work: The authors suggest that display MEs should be treated as arrays of partial MEs for image-to-LaTeX translation. They also highlight the need to align break points in LaTeX code and images when decomposing display MEs. Further research could explore more sophisticated tokenization strategies and leverage the identified textual and visual characteristics for improved ME recognition and translation models.

Summary updated

BibTeX
@inproceedings{daudaravicius2019textual,
  author = {Vidas Daudaravicius},
  title = {{Textual and Visual Characteristics of Mathematical Expressions in Scholar Documents}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2019},
}

Loading related content...

Figure