TL;DR
This paper introduces ATHENA, a novel, fully automatic system for extracting text height and segmenting text lines in old, often degraded, handwritten manuscripts. This is crucial for digital humanities research, enabling more efficient analysis and access to historical documents that are otherwise difficult to process.
What Problem Does This Paper Address?
Digitizing historical manuscripts opens up vast archives for research, but the inherent variability and degradation of these documents pose significant challenges for automated analysis. Unlike modern printed text, old manuscripts can suffer from:
- Degradation: Holes, spots, ink bleed-through, and fading ink.
- Variability: Diverse writing styles, languages, layouts, and inconsistent text height within a single page or even line.
- Complex Layouts: Presence of illustrations, ornaments, and varying text densities.
Existing methods often require manual intervention (e.g., user-defined parameters for text height) or struggle with the noise and irregularities common in these historical documents. This paper aims to provide a robust, parameter-free solution for two fundamental tasks: estimating text height (often referred to as text leading) and segmenting individual text lines.
Methods
ATHENA employs a two-pronged approach:
- Automatic Text Height Estimation:
- Multi-scale Representation: The input image is analyzed at multiple resolutions (levels).
- Frequency-Based Descriptor: For each sub-image at each level, a normalized autocorrelation function (NACF) is computed. The y-axis projection profile of this NACF is then analyzed using the Discrete Fourier Transform (DFT). The dominant spatial frequency in this profile is directly related to the text height.
- Voting Framework: Probability mass functions (PMFs) are generated for each level, and a voting mechanism combines these to produce a robust final text height estimation, effectively averaging out noise and inconsistencies.
- Efficiency: The use of the Plancherel theorem allows for efficient computation of the autocorrelation function in the frequency domain.
- Automatic Text Line Segmentation:
- Coarse Text Region Segmentation: This step leverages the estimated text height and a frequency-based operator to identify potential text areas.
- SIFT Features: Scale-Invariant Feature Transform (SIFT) keypoints are extracted from the image.
- SVM Classifier: A Support Vector Machine (SVM) with a Gaussian radial basis function kernel is trained using SIFT features and the coarse text segmentation to refine the identification of text pixels.
- Modified Projection Profile: A standard projection profile method is adapted to use only the identified text keypoints. Crucially, the bin size of the projection profile is set proportionally to the estimated text height, improving robustness to varying text sizes and densities.
Datasets
The authors evaluated their methods on a large and diverse corpus of:
- 34 Medieval manuscripts
- 19 Arabic handwritten books
This dataset comprises approximately 15,552 pages and over 80,000 text lines. The manuscripts are sourced from various prestigious libraries, including Yale University's Beinecke Library, the Bodleian Library, and others. This diverse collection includes documents with varying resolutions, levels of conservation (ageing, ink bleed-through), languages, and writing styles, making it a challenging benchmark.
Key Insights and Ideas
- Frequency Domain for Text Height: The core insight is that the dominant spatial frequency in the autocorrelation function's projection profile directly correlates with text height, providing a robust measure even in noisy images.
- Multi-scale Analysis for Robustness: Analyzing the image at multiple scales helps to capture text features across different sizes and reduce the impact of local noise or anomalies.
- Text Height as a Key Parameter: The estimated text height is not just an output but a crucial input for the text line segmentation, guiding feature selection and profile binning.
- SVM for Refined Feature Selection: Using an SVM classifier on SIFT features significantly improves the accuracy of identifying text pixels, filtering out noise and non-textual elements.
- Parameter-Free Design: The system is designed to be fully automatic, avoiding the need for manual parameter tuning, which is critical for processing large, heterogeneous collections.
Key Results
- Text Height Estimation Accuracy: The text height estimation achieved a maximum relative error of 15% on a ground truth dataset of 100 images, which the authors deem acceptable given the inherent variability in text height within pages. For the overall dataset of 53 books (15,552 pages), the accuracy is reported to be over 98%.
- Text Line Segmentation Performance: The text line segmentation achieved high Precision and Recall values, consistently above 93% on average across various books. For some books, performance reached 100%.
- Robustness: The methods demonstrated robustness to significant challenges such as ageing, low contrast ink, skewed text, and ink bleed-through.
- Computational Efficiency: The average per-page processing time is around 3 minutes on a multi-core processor, with text height estimation being particularly fast (~6 seconds).
Limitations and Future Work
- Text Height Variability: The authors acknowledge that the accuracy of text height estimation is inversely proportional to character size variability within a page.
- Non-Regular Text Structures: The method may fail when text is not organized in a regular manner, or when inter-line spacing has high variability. This is particularly true for pages with irregular text patterns or additional comments in different styles.
- Single Text Block Assumption: The text line segmentation assumes the page contains only one text block.
- Future Work:
- Investigate more efficient multi-scale approaches, potentially with adaptive domain refinement.
- Explore other local image descriptors (e.g., dense SIFT, DAISY) for improved performance on noisy data.
- Develop a text block segmentation framework to handle multi-column layouts.
- Utilize redundancy across an entire book to further enhance Precision and Recall.
- Conduct more extensive evaluations against state-of-the-art approaches and public benchmarks.
Summary updated