Sublinear information bottleneck based two-stage deep learning approach to genealogy layout recognition

Jianing You, Qing WangFrontiers in Neuroscience 2023 · DOI

TL;DR

Researchers have developed a novel two-stage deep learning framework to automate the digitization of complex Chinese genealogy layouts. By introducing a "Sublinear Information Bottleneck" (SIB), the system effectively filters out visual noise and redundant data, outperforming state-of-the-art models in both accuracy and processing speed.

What Problem Does This Paper Address?

Genealogies are vital cultural artifacts, but their digital preservation is a nightmare for standard layout analysis tools. Ancient Chinese genealogical books feature incredibly complex and diverse structures—some pages are encased in rigid borders, while others are borderless with overlapping text, varying font styles, and significant "noise" from centuries of wear, stains, or poor scanning.

Traditional algorithms like Docstrum struggle to generalize across these different styles. Meanwhile, standard deep learning models often get bogged down by the "curse of dimensionality," where high-dimensional feature maps contain too much redundant information, leading to slower inference and lower accuracy in noisy environments.

Methods

The authors propose a two-stage pipeline powered by a new information-theoretic layer:

  1. Sublinear Information Bottleneck (SIB): This is the core innovation. Based on Information Bottleneck (IB) theory, SIB is designed to find a compressed representation ($T$) of input data ($X$) that preserves only the information most relevant to the target task ($Y$). Unlike standard IB, SIB incorporates second-order statistics to better capture non-linear dependencies in complex datasets.
  2. Stage 1: SIB-ResNet Classifier: The input image first passes through a modified ResNet. This model acts as a gatekeeper, classifying the page layout as either "bordered" or "borderless."
  3. Stage 2: SIB-YOLOv5 Detector: Depending on the classification from Stage 1, the image is sent to a specialized version of the YOLOv5 object detector. The authors embedded SIB layers into the YOLOv5 "CBL" (Convolution-Batch Normalization-Leaky ReLU) modules to force the network to learn more concise, noise-resistant representations of layout components.

Datasets

Because no standard benchmark existed for this specific task, the authors created a custom Genealogy Image Dataset. They scanned and manually labeled common genealogy styles, focusing specifically on the distinction between bordered and borderless inner pages. This dataset served as the ground truth for training both the SIB-ResNet classifier and the SIB-YOLOv5 detector.

Key Insights and Ideas

  • Feature Squeezing: The "bottleneck" isn't just a metaphor. By forcing the network to pass information through a compressed SIB layer, the model is effectively "squeezed," discarding irrelevant pixels (like paper stains or background noise) and focusing on structural markers.
  • The Importance of Borders: Heatmap analysis revealed that the SIB-ResNet model naturally learned to focus on the periphery of the pages. This confirms that the model uses the presence or absence of border lines as the primary signal for classification, mimicking human intuition.
  • Efficiency vs. Accuracy: Typically, adding complexity to a model increases its weight. However, SIB-ResNet achieved higher accuracy (nearly 5% better than GooLeNet) while remaining lightweight, proving that smarter feature selection is more effective than simply adding more layers.

Key Results

The proposed SIB-enhanced models showed significant improvements over standard architectures:

  • Classification: SIB-ResNet achieved a 96.8% mAP and 92.5% Recall, outperforming standard ResNet, VGG, and GooLeNet. It also maintained a fast inference speed of 24ms.
  • Detection: SIB-YOLOv5 converged faster during training and achieved an 89.2% mAP.
  • Qualitative Success: The two-stage approach drastically reduced "localization errors." In borderless genealogies, standard models often hallucinate borders or confuse text blocks; the SIB-optimized pipeline correctly identified these regions by first understanding the global context of the page style.

Limitations and Future Work

While the two-stage approach is highly effective, it currently relies on a binary classification (bordered vs. borderless). The authors acknowledge that Chinese genealogies can be even more diverse. Future work will involve:

  • Expanding Categories: Training the classifier to recognize more than two layout styles.
  • Continuous Optimization: Refining the SIB algorithm to handle even more "extreme" noise cases, such as heavily damaged or faded manuscripts.
  • End-to-End Integration: Potentially moving toward a more unified architecture that doesn't require two distinct stages while maintaining the benefits of the information bottleneck.

Summary updated

Abstract

As an important part of human cultural heritage, the recognition of genealogy layout is of great significance for genealogy research and preservation. This paper proposes a novel method for genealogy layout recognition using our introduced sublinear information bottleneck (SIB) and two-stage deep learning approach. We first proposed an SIB for extracting relevant features from the input image, and then uses the deep learning classifier SIB-ResNet and object detector SIB-YOLOv5 to identify and localize different components of the genealogy layout. The proposed method is evaluated on a dataset of genealogy images and achieves promising results, outperforming existing state-of-the-art methods. This work demonstrates the potential of using information bottleneck and deep learning object detection for genealogy layout recognition, which can have applications in genealogy research and preservation.
BibTeX
@inproceedings{you2023sublinear,
  author = {Jianing You and Qing Wang},
  title = {{Sublinear information bottleneck based two-stage deep learning approach to genealogy layout recognition}},
  booktitle = {Frontiers in Neuroscience},
  year = {2023},
  doi = {10.3389/fnins.2023.1230786},
}
Figure