TL;DR
This paper introduces DocBinFormer, a novel two-level Vision Transformer (ViT) architecture for document image binarization. By effectively capturing both global and local image features through a hierarchical attention mechanism, DocBinFormer achieves state-of-the-art performance on challenging degraded document datasets, outperforming existing methods.
What Problem Does This Paper Address?
Document image binarization is a critical preprocessing step for various document analysis tasks like OCR and information extraction. However, real-world documents often suffer from significant degradation (e.g., aging, ink bleed, uneven lighting, artifacts), making it challenging to accurately separate foreground text from the background. Traditional methods struggle with diverse degradations, and while CNNs are good at local features, they can miss long-range dependencies. Existing ViT models often focus on global features, potentially neglecting crucial local details. This paper aims to bridge this gap by developing a more robust and effective binarization method that can handle complex degradations by leveraging both global context and fine-grained local information.
Methods
The core innovation is the DocBinFormer, a two-level Vision Transformer (TL-ViT) architecture. Unlike standard ViTs that process images as fixed-size patches, DocBinFormer introduces a hierarchical approach:
- Patch and Sub-Patch Tokenization: The input image is first divided into larger patches, and then each patch is further divided into smaller sub-patches.
- Two-Level Transformer Encoder:
- Global Transformer Encoder: Processes the larger patches to capture global contextual information.
- Local Transformer Encoder: Processes the sub-patches to capture fine-grained local details.
- Multi-Head Self-Attention: Both the global and local encoders utilize multi-head self-attention mechanisms to learn dependencies within patches and sub-patches, respectively. This allows the model to adaptively focus on relevant image regions.
- Feature Fusion: The outputs from the global and local encoders are combined to create a rich feature representation that encapsulates both global structure and local texture.
- Transformer Decoder: A standard transformer decoder then reconstructs the binarized output image from the fused features.
Crucially, DocBinFormer achieves this without relying on convolutional layers, processing image data directly through attention mechanisms.
Datasets
The authors trained and evaluated their model on several standard benchmark datasets for document image binarization:
- DIBCO (Document Image Binarization Contest) datasets: DIBCO 2009, 2010, 2011, 2012, 2013, 2014, 2016, 2017, 2018, and 2019.
- H-DIBCO (Handwritten Document Image Binarization Contest) datasets: Used in conjunction with DIBCO datasets.
The evaluation used a leave-one-out cross-validation strategy, where one year's dataset was used for testing, and the rest for training.
Key Insights and Ideas
- Hierarchical Attention is Key: The paper strongly advocates for a hierarchical approach to feature extraction in document binarization. Simply using global attention (like in standard ViTs) or local attention (like in CNNs) is insufficient. Combining them effectively is crucial.
- Sub-Patch Level Detail Matters: The authors highlight that breaking down patches into sub-patches and applying local attention is vital for capturing intricate details that are easily missed by larger receptive fields or global attention alone. This is particularly important for handling variations in text size, font, and subtle degradations.
- Transformer-Native Approach: By avoiding convolutional layers and relying solely on transformer blocks for both global and local feature extraction, DocBinFormer presents a more unified and potentially more powerful architecture for this task.
- Adaptability of Attention: The dynamic nature of attention mechanisms allows them to adapt their focus based on the specific content of the input, making them more robust to diverse degradations compared to fixed receptive fields in CNNs.
Key Results
DocBinFormer demonstrated superior performance across multiple metrics (PSNR, FM, Fps, DRD) on the DIBCO and H-DIBCO datasets.
- Quantitative: The paper reports that DocBinFormer outperforms state-of-the-art methods on most DIBCO datasets. For instance, it achieved the highest PSNR on DIBCO 2009, 2010, 2011, 2012, 2016, and 2017, often by significant margins (e.g., 0.85% and 3.98% absolute improvement on PSNR for DIBCO 2009 compared to the previous best). It was also highly competitive on other datasets.
- Qualitative: Visual results presented in the paper (Figures 2-10) show DocBinFormer producing cleaner binarized images with fewer artifacts and better preservation of text details, especially in severely degraded samples, compared to other methods. Red bounding boxes highlight areas where other models failed, while blue boxes highlight DocBinFormer's successful recovery.
Limitations and Future Work
- Computational Cost: While the paper mentions that creating very small sub-patches (e.g., 2x2) can be computationally expensive, they found a balance with 8x8 sub-patches. Further exploration of hyperparameter tuning, especially patch and sub-patch sizes, could be beneficial.
- Ablation Study: The ablation study in Section IV-E suggests that higher embedding dimensions generally improve results, but the impact of other factors like patch size and layer configurations is explored. The paper found a reasonable balance between performance and computational complexity with their chosen configuration.
- Future Work: The authors suggest that further experimentation with hyperparameter variations and potentially exploring different transformer architectures could lead to even greater improvements. The paper also hints at the potential for this architecture in other low-level vision tasks beyond binarization.
Summary updated