TL;DR
This paper dissects the factors influencing end-to-end deep learning for historical writer identification (HWI), a challenging task due to document degradation and limited data. They found that while end-to-end models struggle with generalization, a specific combination of pre-processing (binarization + Text-AOI selection), a SwinV2-Base backbone, and ArcFace loss achieves state-of-the-art performance with a simpler pipeline.
What Problem Does This Paper Address?
Historical Writer Identification (HWI) aims to determine the author of historical documents. This is significantly harder than modern writer identification due to factors like document aging, ink fading, damage, and the inherent variability in handwriting over time and across individuals. Traditional methods rely on manual feature engineering and are brittle. While end-to-end deep learning promises automation, existing models often fail to generalize to real-world scenarios, especially when test writers are not seen during training (zero-shot learning). This paper investigates why these end-to-end systems falter and identifies key components for robust HWI.
Methods
The authors frame HWI as a pipeline with three core stages:
- Pre-processing: They explore three strategies:
- SIFT-based keypoint detection: A traditional method for extracting local features.
- Historical Document Binarization: Using a deep learning model to create a text mask, separating text from background noise.
- Text-AOI Selection: A novel approach that combines binarization with connected component analysis and dilation to precisely isolate relevant text regions, discarding noise and non-textual elements.
- Feature Extraction (Model Selection & Loss Function):
- Backbones: They compare Transformer models (SwinV2 variants) and CNNs (ResNet18, EfficientNetV2S).
- Loss Functions: Primarily Triplet Loss (to push features from different writers apart and pull features from the same writer together) and ArcFace Loss (for improved feature separation by enforcing angular margins).
- Post-processing:
- Pooling Aggregation: Mean pooling to combine patch-level features into a document-level embedding.
- PCA Dimensional Reduction: To reduce feature dimensionality.
Datasets
The study uses three datasets:
- ICDAR2013-WI: A standard writer identification dataset with well-processed writing lines. Used for baseline performance measurement and comparing architectures.
- ICDAR2017-HistoricalWI: A historical writer identification dataset with document scans. Used for training and testing method comparisons.
- ICDAR2019-HisIR: A challenging historical dataset focusing on document retrieval, featuring raw document scans with significant noise and degradation. This dataset is crucial for evaluating the effectiveness of pre-processing techniques like Text-AOI selection.
Key Insights and Ideas
- End-to-end isn't always better: Many end-to-end deep learning models struggle to generalize in realistic HWI scenarios, particularly zero-shot settings. This is attributed to weak capture of low-level features, inconsistent patch representations, and sensitivity to noise.
- Pre-processing is critical: Simple image resizing is insufficient for noisy historical documents. Combining document binarization with a carefully designed Text-AOI selection method is essential for isolating meaningful writing patterns and improving robustness.
- Text-AOI selection is a game-changer: This technique effectively filters out noise and irrelevant regions, allowing random patch sampling to focus on distinctive writing characteristics, which is crucial for training deep learning models.
- SwinV2 shines: Transformer-based architectures, specifically SwinV2, demonstrate superior performance over CNNs by capturing both local and global patterns in document patches.
- ArcFace Loss for superior separation: While Triplet Loss is a good starting point, fine-tuning with ArcFace Loss significantly improves feature discriminability, leading to better generalization, especially in zero-shot scenarios. This is because ArcFace enforces stricter angular margins, creating more separable representations.
- Simplicity can win: A well-configured pipeline (SwinV2-Base + ArcFace) can achieve state-of-the-art results without overly complex architectures or post-processing.
Key Results
- The proposed end-to-end setup (SwinV2-Base backbone, Text-AOI selection pre-processing, and ArcFace loss for fine-tuning) achieved performance comparable to complex state-of-the-art methods on both standard WI and HWI datasets.
- Specifically, on the ICDAR2017-HistoricalWI dataset, their "Ours" method (SwinV2 + ArcFace) achieved a Top-1 accuracy of 97.15%, P@2 of 96.08%, and mAP of 42.16%, outperforming previous methods.
- PCA dimensional reduction generally did not significantly improve results, suggesting that the learned feature representations are already well-structured.
- SwinV2-Base with cosine Triplet loss and L2 regularization showed the most consistent loss reduction during training.
Limitations and Future Work
- Training Cost: Fine-tuning with ArcFace Loss, while effective, is more resource-intensive and slower than Triplet Loss training.
- Future Work: The authors suggest exploring more efficient fine-tuning strategies to reduce training costs. They also anticipate that the availability of larger historical writer identification datasets will further enable advancements in this field.
Summary updated