TL;DR
This paper introduces the Radial Line Fourier (RLF) descriptor, a novel feature representation designed for robust handwritten word spotting in degraded historical documents. By leveraging log-polar sampling and a concise Fourier transform, RLF offers a fast, short, and noise-resilient alternative to existing descriptors, achieving state-of-the-art performance on challenging datasets.
What Problem Does This Paper Address?
Digitizing and searching historical handwritten documents is a significant challenge due to paper degradation (stains, fading, ink bleed-through), variations in writing styles, and potential language barriers. Traditional word spotting methods, which aim to find specific words within a document image without full transcription, often rely on feature detection and representation. While established descriptors like SIFT and SURF are effective for natural images, their inherent invariance properties can amplify noise in degraded historical documents, leading to poor performance. This paper addresses the need for a feature descriptor specifically tailored for the unique characteristics of historical handwritten text, aiming for improved accuracy and robustness in segmentation-free word spotting.
Methods
The core contribution of this paper is the Radial Line Fourier (RLF) descriptor. The proposed method follows a segmentation-free and training-free approach for word spotting.
- Preprocessing: A simple two band-pass filtering approach is used to remove background noise while preserving crucial gray-level information for feature extraction.
- Keypoint Detection: A combination of four different types of keypoint detectors is employed to capture a diverse range of features in handwritten text: Harris corners, dark/bright blobs (using the Determinant of Hessian), saddle points (negative DoH), and edge detectors. This multi-detector approach aims to capture more comprehensive information than single detectors.
- Radial Line Fourier (RLF) Descriptor: This is the novel component. For each detected keypoint, a log-polar sampling scheme is applied within a circular neighborhood. This transforms the local image region into a square representation (16x16). Then, the Discrete Fourier Transform (DFT) is computed along each radial line. Instead of using the full DFT, only a few elements (specifically, the amplitudes of DFT components for k=2 and k=4) are used to form a compact feature vector of 32 dimensions. This approach is inspired by scale-invariant descriptors but relaxes strict invariance to improve noise robustness.
- Preconditioner-based Feature Matching: To address the issue of false positives arising from similar word parts or overlapping characters, the matching process involves partitioning words into several parts. A nearest neighbor search is then performed within these parts using a deterministic preconditioner. This preconditioner efficiently identifies inliers (true matches) by clustering corresponding matches in a 2D space and discarding outliers, avoiding the need for computationally expensive methods like RANSAC.
Datasets
The paper evaluates its proposed method on the following datasets:
- Barcelona Historical Handwritten Marriages Dataset (BH2M): This dataset consists of historical marriage records from the 17th century, characterized by significant degradation and written by a single author.
- Bentham Dataset: This dataset comprises handwritten documents from the Bentham collection, known for multi-writer variations and document degradations. It was used in two variants, corresponding to the ICFHR 2014 and ICDAR 2015 Handwritten Keyword Spotting Competitions.
Key Insights and Ideas
- Invariance vs. Robustness Trade-off: The paper strongly argues that strict invariance properties (like those in SIFT/SURF) are detrimental for degraded historical documents, as they amplify noise. A "relaxed" feature descriptor that prioritizes robustness to degradation over strict invariance is crucial.
- Log-Polar Sampling for Text: The log-polar transform is well-suited for capturing radial and circular patterns, which can be relevant in handwriting. Combining this with Fourier analysis along radial lines provides a compact and informative representation.
- Concise DFT for Feature Vectors: Using only a few low-frequency components of the DFT significantly reduces the feature vector dimensionality (to 32), leading to faster matching and reduced memory requirements.
- Multi-Detector Keypoints: Employing a diverse set of keypoint detectors (corners, blobs, edges) captures a richer set of local image features, which is beneficial for representing the complex nature of handwriting.
- Part-based Matching for Word Spotting: Dividing words into segments before matching is an effective strategy to mitigate false positives caused by partial word similarities, a common problem in word spotting.
Key Results
The proposed RLF descriptor-based method consistently achieved state-of-the-art performance across the evaluated datasets:
- BH2M Dataset: The RLF method achieved a mean Average Precision (mAP) of 0.783, significantly outperforming existing methods like [Alm12b] (0.513) and [Zag17] (0.530).
- Bentham Dataset (ICFHR 2014): RLF achieved an mAP of 0.490, comparable to the winning algorithm [Kov14] (0.423) and [Zag17] (0.517), and outperforming others like [Ley09] (0.221) and [How13] (0.409).
- Bentham Dataset (ICDAR 2015): RLF achieved an impressive mAP of 0.786, significantly outperforming other methods, including the competition winner [Pra14] and [Zag17] (0.326).
- Feature Representation Comparison: When compared against various feature descriptors (SIFT, SURF, BRISK, ORB, KAZE, DOLF, HoG, Loci, FFT), RLF demonstrated high and consistent performance across datasets, often achieving higher mAP than even advanced descriptors like DOLF and FFT.
Limitations and Future Work
- Bentham Dataset (ICFHR 2014) Challenges: The authors note that the Bentham dataset from the ICFHR 2014 competition presents challenges due to highly variable core text sizes across document pages. This can affect the normalization of text size and potentially lead to information loss. The authors plan to investigate this further.
- Future Work: The authors propose scaling the RLF descriptor's ideas to aid word feature representation for heavily degraded archival databases. They also suggest improvements using query expansion as a future direction.
Summary updated