TL;DR
This paper introduces a novel system for automatically identifying the language of text within complex documents that contain mixed printed and handwritten text, as well as varying layouts. By first distinguishing between writing types and scripts, and then employing a statistical analysis of OCR output, their approach achieves robust language identification, outperforming even Google's language detection tool on challenging real-world document datasets.
What Problem Does This Paper Address?
The automatic identification of language in documents is crucial for downstream tasks like document analysis and information retrieval. However, this problem becomes significantly more challenging when dealing with "heterogeneous and complex documents." These documents often feature a mix of printed and handwritten text, different scripts (like Latin and Arabic), and varied layouts. Existing language identification systems typically perform well on clean, digital text but struggle with the noise, variability, and mixed content found in scanned documents. Specifically, accurately identifying the language of a document image requires first understanding how the text is written (printed vs. handwritten) and what script it uses, as these factors heavily influence recognition and analysis. This paper tackles this multi-faceted problem by proposing a unified system that addresses all three aspects: writing type, script, and language identification.
Methods
The proposed system operates in a sequential manner:
- Segmentation: Text blocks are first extracted from the document image.
- Writing Type Identification: This stage determines if a text block is printed or handwritten. It relies on analyzing the connected components (CCs) within the text block, specifically using a codebook of contour fragments.
- Script Identification: Building upon the writing type information, this step identifies the script (e.g., Latin, Arabic). Similar to writing type identification, it uses a codebook-based approach on CC contour fragments, but with specialized codebooks tailored to the identified writing type.
- Language Identification: For Latin scripts, this is the most complex part. Since physical features alone are insufficient to distinguish between languages like French and English, the system employs a statistical text analysis approach. This involves using an OCR engine (specifically, a language-free HMM-based OCR) to transcribe the text blocks. Language profiles are then estimated based on character and n-gram (bi-gram and tri-gram) distributions from these transcriptions. The system compares the document's profile to pre-computed language profiles (French-hand, French-printed, English-hand, English-printed) using a weighted chi-squared score to determine the language.
The core novelty lies in the integrated approach, where writing type and script identification are performed using a robust contour fragment-based method, and this information is then leveraged to improve the subsequent language identification, which relies on statistical analysis of OCR output.
Datasets
The primary dataset used for training and evaluation is the MAURDOR dataset. This dataset is described as heterogeneous, containing documents like forms, business documents, correspondence, maps, and newspapers, with mixed printed and handwritten texts in French, Arabic, and English. The paper mentions that the MAURDOR dataset was used for evaluation campaigns, with the authors comparing their results to participants of the second campaign (November 2013). The dataset consists of 6000 training documents and two evaluation sets of 1000 documents each.
Key Insights and Ideas
- Hierarchical Approach is Crucial: The paper strongly advocates for a hierarchical approach where identifying the writing type and script before attempting language identification is essential for handling complex, mixed-content documents.
- Contour Fragments as Powerful Features: The use of contour fragments from connected components proves to be an effective method for distinguishing between writing types and scripts, even in challenging scenarios. This approach is robust to variations in text size and layout.
- OCR Errors Can Be Informative: For language identification, especially when dealing with languages sharing the same alphabet, the system leverages statistical analysis of OCR output. Interestingly, the paper suggests that the errors introduced by the OCR can also be informative for distinguishing languages, particularly when dealing with Arabic documents where only a Latin OCR is used.
- Block-Level Analysis is Challenging: Identifying language at the individual text block level is significantly harder than at the document level, especially for small blocks, due to limited textual information.
Key Results
- Writing Type Identification: Achieved an accuracy of 93.50% on the second MAURDOR campaign, outperforming other participants in the campaign.
- Script Identification: Achieved an accuracy of 92.51% on the second MAURDOR campaign.
- Language Identification: The "Code + distrib 2G" system (using codebooks for script and bi-grams for language) achieved an accuracy of 87.36% on the second MAURDOR campaign.
- Comparison with Google Plug-in: The proposed system, even without access to ground-truth transcriptions, achieved comparable or better performance than the Google plug-in when evaluated on the MAURDOR dataset, especially at the block level. At the page level, the Google plug-in performed better but still lagged behind the authors' system when evaluated on OCR output rather than ground truth.
Limitations and Future Work
- Preprocessing for Contour Fragments: The authors suggest that improving the quality of contour fragments by adding a preprocessing step to remove rule lines and enhance fragment quality could further improve performance.
- OCR Strategy: The current language identification system uses a language-free OCR. The authors propose exploring OCR engines that incorporate language models for French and English, allowing for a comparison of recognition scores to aid language selection.
- Broader Language and Script Support: The current system is evaluated for French, English, and Arabic. Future work should involve evaluating the system on datasets with more scripts and languages to assess its generalizability.
Summary updated