Deep Learning for Word-Level Handwritten Indic Script Identification

Soumya Ukil, Swarnendu Ghosh, Sk Md Obaidullah, Kc Santosh, Kaushik Roy, Nibaran DasRTIP2R 2018 · OpenReview

TL;DR

This paper tackles the challenging problem of identifying handwritten Indic scripts within documents, a crucial step for accurate Optical Character Recognition (OCR). They propose a novel deep learning approach that combines Convolutional Neural Networks (CNNs) with multi-level wavelet transforms to extract robust features, achieving a state-of-the-art identification rate of 94.73% across 11 different Indic scripts.

What Problem Does This Paper Address?

Optical Character Recognition (OCR) systems are essential for digitizing text, but they are highly script-dependent. In multilingual regions like India, documents often contain multiple scripts, making script identification a prerequisite for accurate OCR. Traditional methods rely on hand-crafted features, which can be time-consuming to design and may not capture the full complexity of handwritten scripts. This paper addresses the need for a more automated and effective script identification system, specifically for the diverse set of Indic scripts, by leveraging the power of deep learning.

Methods

The core of this paper's contribution lies in its innovative feature extraction and selection strategy. Instead of relying on hand-crafted features, they employ Convolutional Neural Networks (CNNs) to learn discriminative features directly from the image data.

Here's a breakdown of their approach:

  • Multi-level Feature Extraction: They go beyond standard spatial domain representations by incorporating the multilevel 2D discrete Haar wavelet transform. This allows them to capture both spatial and frequency domain information from the input images.
  • Scaled Representations: Images are scaled to various sizes (32x32, 48x48, 128x128) to ensure that features are extracted at different resolutions.
  • Ensemble of CNNs: Ten different CNN architectures are trained, each specializing in extracting features from specific representations (spatial or wavelet-transformed) and at different scales. Each CNN outputs a set of 1024 features.
  • Feature Concatenation: The features from these ten CNNs are concatenated to form a comprehensive feature vector of 10240 dimensions.
  • Classification: A Multi-Layer Perceptron (MLP) is used as the final classifier to identify the script based on the combined feature vector.

The novelty lies in the synergistic combination of wavelet transforms for richer feature representation and an ensemble of CNNs for robust feature selection, moving away from manual feature engineering.

Datasets

The primary dataset used in this study is PHD_Indic_11.

  • PHD_Indic_11: This dataset comprises 11,000 scanned word images (grayscale) from 11 different Indic scripts, with 1,000 words per script. The authors note that this dataset is significant because larger datasets for Indic script identification were not readily available at the time of their research.
  • Training/Testing Split: A conventional 4:1 ratio was used, with 8,800 images for training and 2,200 for testing.

Key Insights and Ideas

  • Wavelets Enhance Feature Richness: Incorporating wavelet transforms, particularly the Haar wavelet, alongside spatial domain features significantly improves the ability of CNNs to capture subtle script characteristics. This suggests that frequency domain information is crucial for distinguishing between similar-looking scripts.
  • Ensemble Learning for Robustness: Using an ensemble of multiple CNNs, each trained on different image scales and representations, leads to a more robust and comprehensive feature set. This approach mitigates the limitations of any single CNN architecture.
  • Deep Learning Outperforms Hand-crafted Features: The paper strongly advocates for deep learning's ability to automatically learn discriminative features, surpassing the performance of traditional hand-crafted feature methods for this complex task.
  • Scale Matters: Experimentation with different image scales (32x32, 48x48, 128x128) and their impact on accuracy highlights the importance of multi-scale analysis in script identification.

Key Results

The proposed method achieved a maximum script identification rate of 94.73% using the concatenated features from ten CNNs and an MLP classifier.

  • Individual CNN Performance: The best performing individual CNN was CNNs,128,3 which achieved a script identification rate of 90%.
  • Ensemble Performance: Combining CNNs for spatial representation (CNNs) achieved 94.14%, while frequency domain representations (CNNf) reached 90.27%. The combined approach of spatial and frequency domain representations yielded the highest accuracy of 94.73%.
  • Comparative Performance: The paper's method significantly outperforms previous state-of-the-art techniques, including:
  • Obaidullah et al. [6] (hand-crafted features): 91.00% (outperformed by 4.73%)
    • LeNet [7] (CNN): 82.00% (outperformed by 12.73%)
    • AlexNet [9] (CNN): 92.14% (outperformed by 2.59%)

Limitations and Future Work

  • Limitations:
    • The paper acknowledges that they did not achieve 100% script identification accuracy, providing examples of misclassified samples (Fig. 6).
  • While the PHD_Indic_11 dataset is noted as the largest available at the time, the authors implicitly suggest that even larger and more diverse datasets could further improve performance.
  • Future Work:
    • The paper's success with multi-scale analysis and wavelet transforms suggests that exploring other advanced wavelet families or more sophisticated feature fusion techniques could be beneficial.
  • Investigating more complex deep learning architectures beyond standard CNNs and MLPs might yield further improvements.
  • Extending the approach to handle even more complex scenarios, such as noisy images, degraded text, or a larger number of scripts, would be a logical next step.

Summary updated

Abstract

In this work, we present a novel method that uses convolutional neural networks (CNNs) for multi scale and multi spectral feature extraction. The model is not only limited to conventional spatial domain representation but also multilevel two dimensional discrete Haar wavelet transform, where image representations are scaled to a variety of different sizes. These are then used to train different CNNs to select features. To be precise, we use 10 different CNNs that select a set of 10240 features, i.e. 1024/CNN. With this, 11 different handwritten scripts are identified, where 1000 words per script are used. In our test, we have achieved a maximum script identification accuracy of 94.73% using multi-layer perceptron (MLP). Our results outperform several state of the art techniques.
BibTeX
@inproceedings{ukil2018deep,
  author = {Soumya Ukil and Swarnendu Ghosh and Sk Md Obaidullah and KC Santosh and Kaushik Roy and Nibaran Das},
  title = {{Deep Learning for Word-Level Handwritten Indic Script Identification}},
  booktitle = {RTIP2R},
  year = {2018},
  url = {https://openreview.net/forum?id=a95cwFfHL2},
}
Figure