Convolutional Neural Networks for Page Segmentation of Historical Document Images

Kai Chen, Mathias Seuret, Jean Hennebert, Rolf IngoldarXiv 2017 · DOI · arXiv

Abstract

This paper presents a page segmentation method for handwritten historical document images based on a Convolutional Neural Network (CNN). We consider page segmentation as a pixel labeling problem, where each pixel is classified into predefined classes. Traditional methods rely on hand-crafted features tuned with prior knowledge. In contrast, we propose learning features directly from raw image pixels using a simple CNN with only one convolutional layer. Despite its simplicity, this architecture achieves competitive results against deeper models on various public datasets, demonstrating the effectiveness and superiority of the proposed approach.

Introduction

Page segmentation—the process of partitioning a document image into regions such as text, decorations, and margins—is a critical precursor to Optical Character Recognition (OCR) and automated document understanding. While modern printed documents are relatively straightforward to segment, historical manuscripts present significant challenges. These include complex layouts with non-linear text lines, physical degradation like ink bleed-through or stains, and high variability in scribal styles and decorative elements.

Traditional segmentation methods typically rely on "hand-crafted" features, such as Gabor filters or Voronoi diagrams, coupled with rigid rule-based systems. These approaches are often brittle, difficult to tune, and fail to generalize across diverse manuscript collections. This research demonstrates that a surprisingly simple CNN can outperform these complex, hand-tuned models by learning features directly from raw pixels.

Methodology

The authors frame page segmentation as a pixel-labeling task. To maintain computational efficiency while processing high-resolution document images, they employ a two-step strategy involving superpixel preprocessing and a minimalist neural architecture.

Superpixel Preprocessing

Instead of classifying every pixel independently, the system uses the Simple Linear Iterative Clustering (SLIC) algorithm to group pixels into "superpixels." The CNN classifies the center pixel of a 28x28 patch surrounding each superpixel, and the resulting label is assigned to the entire group. This approach significantly reduces the number of forward passes required per page.

The Minimalist CNN Architecture

In an era of increasingly deep networks, this study explores the efficacy of a shallow model. The architecture consists of:

  • Input Layer: A 28x28 grayscale image patch.
  • Convolutional Layer: A single layer with 4 kernels of size 3x3.
  • Fully Connected Layer: 100 neurons utilizing ReLU activation.
  • Output Layer: A Softmax layer predicting the probability of the patch belonging to specific classes (e.g., main text, comments, decorations).
The proposed simple CNN architecture featuring a single convolutional layer and one hidden layer.

Training is performed using Stochastic Gradient Descent (SGD) with a 0.5 dropout rate to mitigate overfitting, a crucial consideration given the relatively small size of historical document datasets.

Datasets

The model was evaluated on six diverse public datasets:

  • G. Washington: English manuscripts with a relatively consistent layout.
  • St. Gall & Parzival: Latin and German manuscripts on parchment, provided in color.
  • CB55, CSG18, & CSG863: Challenging benchmarks featuring complex layouts and multiple languages (Italian and Latin) from the 11th and 14th centuries.

Results and Discussion

The experimental results indicate that "shallow" architectures are remarkably effective for the textures found in historical manuscripts.

Performance Metrics

The simple CNN consistently outperformed state-of-the-art methods, including Multi-Layer Perceptrons (MLP) and Conditional Random Fields (CRF). On the St. Gall dataset, the CNN achieved a pixel accuracy of 98%. Furthermore, the method is highly efficient, processing a document image in approximately one second—2x to 5x faster than previous CRF-based models.

Quantitative performance comparison and qualitative segmentation results across different datasets and methods.

Key Insights

  • Shallow vs. Deep: Adding more convolutional layers or kernels did not significantly improve performance and occasionally led to degradation due to overfitting on smaller datasets like G. Washington.
  • Spatial Sensitivity: The authors found that max pooling offered little benefit. In page segmentation, the exact spatial location of features is vital; the network needs to know precisely where ink resides to label pixels accurately.
  • End-to-End Efficiency: By integrating feature extraction and classification, the model avoids the information loss inherent in pre-defined feature sets.

Limitations

While effective, the model's accuracy is inherently tied to the quality of the initial superpixel clustering. If a superpixel spans a boundary between two classes (e.g., text and background), the labeling will be imperfect. Additionally, the sensitivity to training data volume suggests that while shallow networks are robust, they still require careful tuning for very small datasets.

Conclusion

This research proves that for specific computer vision tasks like historical document segmentation, massive "deep" networks are not always necessary. A minimalist CNN with a single convolutional layer can provide superior accuracy and speed, offering a practical and robust solution for the digital humanities. Future work may involve integrating this local patch-based classifier into a global context-aware system to further improve results on extremely noisy or ambiguous pages.

Summary updated

Abstract

This paper presents a page segmentation method for handwritten historical document images based on a Convolutional Neural Network (CNN). We consider page segmentation as a pixel labeling problem, i.e., each pixel is classified as one of the predefined classes. Traditional methods in this area rely on hand-crafted features carefully tuned considering prior knowledge. In contrast, we propose to learn features from raw image pixels using a CNN. While many researchers focus on developing deep CNN architectures to solve different problems, we train a simple CNN with only one convolution layer. We show that the simple architecture achieves competitive results against other deep architectures on different public datasets. Experiments also demonstrate the effectiveness and superiority of the proposed method compared to previous methods.
BibTeX
@misc{chen2017convolutional,
  author = {Kai Chen and Mathias Seuret and Jean Hennebert and Rolf Ingold},
  title = {{Convolutional Neural Networks for Page Segmentation of Historical Document Images}},
  year = {2017},
  eprint = {1704.01474},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1704.01474},
}
Figure