Automatic document orientation detection and categorization through document vectorization

Shijian Lu, Chew Lim TanACM Multimedia 2006 · DOI

TL;DR

This paper introduces a novel method for automatically detecting the orientation and identifying the language of document images. By transforming documents into "document vectors" based on the distribution of vertical component runs, the system achieves high accuracy in both tasks, simplifying document processing pipelines.

What Problem Does This Paper Address?

In today's digital world, vast amounts of documents are scanned, digitized, or captured via cameras. These documents often come in various orientations (e.g., upside down, rotated) and can be in multiple languages. Manually correcting the orientation and identifying the language of each document is a time-consuming and expensive process, especially for large digital archives. This paper tackles the challenge of automating both document orientation detection and language categorization, aiming to reduce human intervention and improve the efficiency of document processing workflows.

Methods

The core of the proposed method lies in "document vectorization." This technique transforms a document image into a numerical representation, a "document vector," that captures both its orientation and language characteristics.

Here's a breakdown of the key steps:

  • Preprocessing: Documents undergo initial processing, including binarization (converting to black and white), noise removal, and skew correction. The skew angle is assumed to be within ±90 degrees at this stage, meaning documents can be either correctly oriented or upside down.
  • Vertical Component Run (VCR) Extraction: The system analyzes the density and distribution of vertical component runs (VCRs) within the text. A VCR is identified when a vertical scan line encounters a character component. The position of these VCRs is categorized into three zones: upper, middle, and base.
  • Document Vector Construction: Each document is represented by a 32-dimensional vector. The first 8 dimensions capture the density of VCRs, while the remaining 24 dimensions encode the positions of VCRs within the three text zones. This vector effectively encodes information about the document's language (due to character structure differences) and its orientation (how VCRs appear in different zones when upside down).
  • Vector Template Construction: For each language of interest, a pair of "vector templates" is created. These templates represent the average document vectors for that language in both correct and upside-down orientations, derived from a set of training documents.
  • Orientation Detection and Categorization: A query document's vector is compared to the pre-constructed vector templates using the Bray Curtis distance. The orientation and language of the query document are then assigned based on the template with the smallest distance.

Datasets

The paper utilizes a dataset of 492 testing document images for evaluation. These documents are printed in four different languages: Arabic, Chinese, Korean, and Roman. For training, 120 documents were used, with 30 documents per language. These training documents were scanned at 600 ppi with controlled skew angles close to zero. The paper also mentions an experiment with 60 Bangla documents (20 for training, 40 for testing) to demonstrate the system's extendability.

Key Insights and Ideas

  • VCRs as a Universal Feature: The paper demonstrates that the density and distribution of Vertical Component Runs (VCRs) can effectively capture both language-specific character structures and orientation information, even across diverse scripts like Arabic, Chinese, Korean, and Roman.
  • Document Vectorization for Multi-task Learning: By encoding document features into a unified vector space, the system elegantly handles both orientation detection and language categorization simultaneously, simplifying the overall processing pipeline.
  • Robustness to Arbitrary Skew: The method is designed to handle documents with arbitrary skew angles within a certain range, making it practical for real-world scanned documents.
  • Extendability to New Languages: The framework is designed to be easily extended to new languages by simply creating new vector templates from training data, highlighting its flexibility.

Key Results

The proposed method achieves impressive results on the tested dataset:

  • Average Orientation Detection Accuracy: Up to 97.56%
  • Average Document Categorization Accuracy: Up to 99.59%

The paper also notes that the performance is significantly affected by the number of text lines in a document. For documents with a large number of text lines, the accuracy is high. However, for documents with only one or two text lines, the performance deteriorates.

Limitations and Future Work

The authors acknowledge the following limitations and suggest future research directions:

  • Performance Degradation with Short Documents: The method's accuracy drops significantly for documents with very few text lines (e.g., one or two). This is because the document vector may not accurately reflect the true density and distribution of VCRs.
  • Language Discrimination Challenges: For languages with very similar VCR density and distribution to those already in the system, the proposed method might struggle to differentiate them.
  • Future Work:
    • Developing character-level orientation detection and categorization techniques for short documents.
  • Exploring other shape statistics to compensate for VCR inefficiency in distinguishing certain languages.
  • Investigating the problem of document orientation detection and categorization in future work.

Summary updated

BibTeX
@inproceedings{lu2006automatic,
  author = {Shijian Lu and Chew Lim Tan},
  title = {{Automatic document orientation detection and categorization through document vectorization}},
  booktitle = {ACM Multimedia},
  year = {2006},
  doi = {10.1145/1180639.1180673},
}
Figure