TL;DR
This paper introduces a novel algorithm for digitizing printed tables into an Excel format, preserving cell relationships. By combining image processing with OCR, it tackles the challenge of converting paper-based data into a structured digital format, crucial for digital transformation initiatives.
What Problem Does This Paper Address?
In today's push for digital transformation (DX), a significant amount of valuable data remains locked away in printed documents, particularly in tables. While Optical Character Recognition (OCR) can digitize text, tables present a unique challenge. The inherent structure of tables, with characters separated by rows and columns into cells, makes it difficult for standard OCR to maintain the relationships between these cells. This loss of relational information renders the digitized data less useful. The paper aims to solve the problem of converting printed tables into a structured Excel format that accurately preserves the relationships between cells.
Methods
The proposed algorithm employs a two-pronged approach: image processing for structural analysis and OCR for character recognition.
- Image Processing (OpenCV):
- The process begins with scanning a printed table into an image.
- OpenCV is used to detect and extract table lines (both horizontal and vertical) by applying erosion and dilation operations with specific kernels.
- These lines are combined to form a table frame.
- The
findContoursfunction identifies the boundaries of individual cells. boundingRectis used to get the rectangular coordinates of each cell.- Cells are then sorted based on their positions to establish a row-by-row and column-by-column order.
- A crucial step involves creating a tree structure to represent the topology of cells within each row, capturing their relationships.
- OCR (Tesseract):
- Once cells are identified and isolated, Tesseract OCR is applied to each cell to recognize the characters within.
- The
pytesseractPython wrapper is used to integrate Tesseract into the workflow. - Excel Output (openpyxl):
- Finally, the recognized text from each cell, along with the established cell topology, is used to construct an Excel file.
- The
openpyxllibrary is employed to create and manipulate the Excel workbook, including merging cells to reflect the original table structure.
The algorithm is specifically designed to handle two common table layouts: a simple grid where rows and columns are consistent, and a slightly more complex layout where the number of rows might vary across columns.
Datasets
The paper does not introduce a new dataset. Instead, it evaluates its proposed algorithm on:
- 19 scanned table images.
- 17 screenshotted table images obtained from websites.
These images were selected because they adhere to the table layout features described in Section 4.1 of the paper.
Key Insights and Ideas
- Topology is Key: The core innovation lies in explicitly modeling the topology of table cells using a tree structure. This goes beyond simple cell extraction and is essential for reconstructing the relational integrity of the table.
- Image Processing for Structure, OCR for Content: The paper effectively demonstrates a synergistic approach where image processing handles the structural understanding of the table, and OCR focuses on extracting the textual content within each identified cell.
- Handling Table Layout Variations: By considering two common table layouts, the algorithm shows an attempt to address some of the inherent diversity in table structures, moving beyond a one-size-fits-all approach.
- Limitations of Standard OCR: The evaluation section highlights that even with structural information, standard OCR tools like Tesseract struggle with certain characters (subscripts, Greek letters) and image quality issues (blurry scans, noise), indicating areas for future improvement.
Key Results
- Table Structure Recognition: The algorithm achieved 100% accuracy in recognizing the table cell structure for all tested images.
- Character Recognition: The accuracy of character recognition (p) was calculated using a formula that accounts for recognized characters versus total characters. While the exact overall percentage isn't a single headline number, the paper presents bar charts showing rates for scanned and screenshotted images. For scanned images, several fell below 90%, with specific examples showing rates around 84.66% and 82.13%. The paper considers forms with p > 80% as having a "qualified recognition effect."
- Comparison with Microsoft Office: The paper notes that Microsoft Office's scanning application sometimes failed to recognize complex table structures, producing incorrect grids or unnecessary cells, whereas the proposed algorithm successfully extracted the correct structure in those cases.
Limitations and Future Work
- OCR Limitations: The paper explicitly acknowledges the limitations of the adopted OCR software (Tesseract), including difficulties with small cell sizes (border recognition), vertical text, subscripts, Greek letters, and blurry scanned images with noise.
- Table Layout Complexity: The current algorithm is limited to two specific table layouts. More complex table structures are not handled.
- Future Work: The authors propose several avenues for improvement:
- Enhancing character recognition accuracy by refining the OCR software.
- Extending the algorithm to handle more complex table structures.
- Applying the algorithm to a wider variety of table images.
- Directly identifying more complex table types than currently supported.
Summary updated