Table Recognition and Recovery in the Camera-Captured Image with Complex Background

Guiyang Xie, Yirong Mao, Zifan Nan, Shizhe Wang, Yue Li, Wei WangAdvances 2015 · DOI

Abstract

With the development of OCR techniques, researchers have increasingly utilized image processing methods to analyze image-type documents—typically scanned images—to extract character and layout information. However, tables have received relatively less attention in OCR despite being critical structures for organizing classified information. This paper presents an innovative method designed to detect, recognize, and recover tables from camera-captured images featuring complicated backgrounds. The pipeline includes image preprocessing, binarization, edge detection, cell detection, and structural analysis. Experimental results using Emgu CV demonstrate high accuracy and efficiency.

Introduction

While Optical Character Recognition (OCR) has reached high levels of accuracy for standard text, table recognition remains a significant challenge. Most existing business OCR systems are optimized for high-quality flatbed scans. In real-world scenarios, however, documents are frequently captured using smartphone cameras, which introduces three major hurdles:

  1. Perspective Distortion: Tables appear as trapezoids rather than rectangles due to camera angles.
  2. Uneven Lighting: Shadows and highlights cause standard global binarization to fail, often breaking thin table lines.
  3. Complex Backgrounds: Watermarks or textured paper can be mistaken for text or table borders.

Methodology

The proposed system follows a multi-stage pipeline implemented using Emgu CV (a .NET wrapper for OpenCV). The process begins with image input and proceeds through rectification, binarization, edge detection, morphological processing, cell detection, and finally, table recovery.

System Flowchart

Image Rectification

To correct perspective tilt, the system identifies the four corners of the document and applies a projective warp (homography matrix). This transforms the distorted trapezoidal view back into a flat, rectangular coordinate system.

Adaptive Binarization

Instead of a single global threshold, the authors utilize Wolf’s Method for binarization. This technique calculates a threshold for each pixel based on a local sliding window, making it significantly more robust against the uneven luminance typical of camera-captured imagery.

Mathematical Morphology and Edge Detection

To clean up noise, the system employs Canny edge detection followed by morphological operations. Specifically, two structural elements—seH (horizontal) and seV (vertical)—are used to isolate and "heal" the straight lines of the table grid while filtering out non-table artifacts and background noise.

Results and Analysis

The authors evaluated their system on a set of 15 experimental camera-captured images featuring complex backgrounds and varying lighting conditions. The accuracy was measured by calculating the coincident area between the reconstructed table cells and the original image cells.

Processing Sequence

Out of 15 test cases, most achieved high accuracy, with several scores exceeding 95%. While one outlier performed as low as 43% due to extreme noise, the qualitative results demonstrate a clear and effective progression from a distorted, shadowed photo to a clean, digital grid. The system even includes logic to handle empty cells by inheriting formatting from neighboring cells during the reconstruction process.

Conclusion

This paper presents a robust pipeline for extracting structured table data from handheld camera photos. By combining projective geometry for perspective correction with adaptive binarization and morphological filtering, the system successfully recovers tables even in challenging environments. Future work will focus on automating the initial corner detection and handling more complex layouts, such as merged or borderless cells.

Summary updated

Abstract

With the development of OCR techniques, people start to use image processing methods to analyze and process the image-type documents (typically scanned images) for obtaining the information of characters and the layout. On the other hand, table, draw relatively less attention in OCR even it is an important writing structure containing plenty of classified massages in special format. This paper comes up with an innovative method aiming to detect, recognize and recover the table on a camera-capture image with complicated background. This method mainly includes the preprocessing, binarization and edge detection of the image, together with the detection of cells, as well as the analysis and recovery of table structure. In practical experiments, Emgu CV is employed to achieve more accurate result with better efficiency.
BibTeX
@inproceedings{xie2015table,
  author = {Guiyang Xie and Yirong Mao and Zifan Nan and Shizhe Wang and Yue Li and Wei Wang},
  title = {{Table Recognition and Recovery in the Camera-Captured Image with Complex Background}},
  booktitle = {Advances},
  year = {2015},
  doi = {10.2991/asei-15.2015.27},
}
Figure