TL;DR
This paper introduces the largest and most diverse Bangla OCR dataset to date, comprising over 4 million human-annotated images across various document types and handwriting styles. This comprehensive resource aims to significantly advance Bangla OCR technology by addressing the scarcity of high-quality labeled data, paving the way for more accurate and robust systems.
What Problem Does This Paper Address?
Optical Character Recognition (OCR) has revolutionized data extraction, but its application to languages with complex scripts and diverse writing styles, like Bengali, has been hampered by a lack of comprehensive datasets. Existing Bangla OCR efforts have often focused on isolated characters or specific document types, leaving a significant gap for robust, end-to-end OCR systems that can handle the full spectrum of real-world Bangla text, including computer-generated text, various print styles, and diverse handwriting. This paper tackles this challenge by creating a "gold standard" dataset that aims to be the foundation for developing advanced Bangla OCR.
Methods
The authors developed a meticulous data collection and annotation pipeline. Key aspects include:
- Diverse Data Sourcing: They collected data from a wide array of sources, including computer-composed documents (books, official notices), letterpress and typewriter prints, handwritten texts from over 150 individuals, and outdoor scene text (banners, nameplates).
- Custom Tool Development: Recognizing the lack of pre-existing tools, the team built custom software for:
- Cropping: Tools for segmenting words and characters from images. This involved both contour-based methods for structured documents and manual cropping for handwritten and outdoor text.
- Annotation: A web-based tool and a mobile application for annotating cropped images with their corresponding text.
- Controlled Annotation Procedure: A rigorous three-step annotation and one-step validation process was implemented. Multiple annotators worked on each data point, with supervisors validating their work to ensure adherence to "gold standard" criteria.
- Quality Assessment: Statistical measures like Fleiss' Kappa were used to assess inter-annotator reliability, with scores indicating high agreement (e.g., 0.91 for computer-composed documents).
Datasets
The primary contribution is the newly introduced Bangla OCR corpus, which contains over 4 million human-annotated images. This dataset is unique in its:
- Scale: Over 4 million annotated words and characters.
- Diversity: Encompasses computer-composed text, letterpress, typewriter prints, handwritten documents (isolated and running words), and outdoor scene text.
- Quality: Achieved through a rigorous, multi-stage human annotation and validation process.
The paper also references existing datasets for English OCR (e.g., Uber-Text, RoadText-1K, TextOCR) and some prior Bangla OCR datasets (e.g., BanglaLekha-Isolated, EkushNet, Borno) to highlight the limitations of previous work.
Key Insights and Ideas
- The critical role of diverse data: The paper strongly emphasizes that robust OCR, especially for complex scripts, requires a dataset that mirrors real-world variations in document types, writing styles, and image quality.
- Human annotation is indispensable: Despite advancements in automated methods, the creation of a "gold standard" dataset necessitates meticulous human effort for accuracy and reliability, particularly for handwritten and noisy text.
- Custom tools are essential for specialized tasks: The development of bespoke software for cropping and annotation was crucial for handling the unique challenges of Bangla text and diverse data sources.
- Validation is key to dataset quality: The multi-stage validation process, including inter-annotator agreement checks, is vital for ensuring the dataset's integrity and usefulness.
Key Results
- Dataset Creation: The successful development and annotation of a large-scale, diverse Bangla OCR dataset (over 4 million images).
- Quality Metrics: High inter-annotator agreement scores (Kappa scores ranging from 0.78 to 0.93) demonstrate the reliability of the annotation process.
- Model Performance (Preliminary): While the dataset is primarily for training, preliminary evaluation using a subset with a CRNN-VDS model showed promising results, with Character Recognition Rates (CRR) reaching up to 93.04% for computer-composed documents.
Limitations and Future Work
- COVID-19 Impact: The pandemic significantly challenged data collection, coordination, and contracting with participants, potentially impacting the timeline and scope.
- Initial Tooling Issues: The early mobile application for annotation faced complications, requiring iteration and the development of more robust web tools.
- Limited Access: Due to confidentiality, only a small subset of the data was available for external evaluation, limiting immediate widespread testing.
- Future Work: The authors anticipate that this dataset will be instrumental in advancing Bangla OCR research, enabling the training of more sophisticated models and improving language understanding and accessibility in the digital realm. They suggest that researchers can leverage this corpus for various needs, including document type, word, and character-level analysis.
Summary updated