Table Extraction Using Spatial Reasoning on the CSS2 Visual Box Model

Wolfgang Gatterbauer, Paul BohunskyAAAI 2006

This paper, presented by researchers at the Vienna University of Technology in 2006, proposes a paradigm shift in web data extraction. Rather than parsing the often-convoluted HTML source code, the authors introduce VENTrec (Visualized Element Node Table recognition), a system that interprets web pages based on their visual layout as rendered by a browser engine.

The Problem: HTML's Non-Injective Mapping

Extracting structured data from the web is difficult because HTML was designed for visual presentation, not semantic storage. A <table> tag might be used for layout rather than data, while a legitimate data table might be constructed using nested <div> tags or lists. This creates a "non-injective mapping" problem: many different code implementations can result in the same visual appearance. The authors argue that the intended meaning of a table is found in its visual alignment and spatial syntax, which humans naturally perceive, rather than its underlying tags.

Methodology

The CSS2 Visual Box Model

VENTrec utilizes a browser engine (Mozilla) to render the page and retrieve the exact $(x, y)$ coordinates and dimensions of every element. This transforms the document from a tree of tags into a collection of rectangular boxes. To manage these boxes, the system superimposes a "Double Topographical Grid" to track cardinal directions and relative positions.

Spatial Relations and "Twin Neighbors"

The core logic relies on categorizing the relationship between adjacent boxes using a modified version of Allen’s interval relations. The most critical relation is the Twin Neighbor: adjacent boxes that align perfectly along one axis.

Spatial relations between boxes based on Allen’s interval logic, defining the 'Twin Neighbor' heuristic.

By identifying these twins, the system can determine which elements belong to the same logical row or column, regardless of the HTML tags used to create them.

The Circulating Expansion Algorithm

To reconstruct a table, VENTrec uses a recursive, "circulating" algorithm. It begins with a "seed" (a keyword of interest) and identifies the box containing it. The algorithm then expands outward in a clockwise spiral (right, down, left, up), merging adjacent twin boxes into a larger hyperBox.

The circulating expansion algorithm steps: starting from a seed, expanding through adjacent twins, and outputting the final bounding rectangle.

The expansion continues until it encounters a boundary where alignment breaks (a "step neighbor") or the boxes no longer match the established grid logic. A region is validated as a Table of Interest (TOI) only if it is two-dimensional (at least 2x2) and contains no overlapping boxes.

Evaluation and Results

The authors evaluated VENTrec using three datasets: 100 Google search results, 14 linked web tables, and a subset of 50 pages from the Wang & Hu (2002) ground truth dataset.

Despite being a purely deterministic approach without machine learning, the system proved remarkably robust:

  • High Accuracy: The spatial reasoning approach achieved an F-measure of 90.2%.
  • Heuristic Optimization: By adding a simple rule to ignore cells with more than 20 words (filtering out paragraphs), the Precision rose to 96.7%, with an overall F-measure of 92.8%.
  • Performance: On 2006-era hardware (1.6 GHz Pentium M), the algorithm typically processed pages in less than one second.

Key Insights and Limitations

VENTrec's primary strength is its tag agnosticism. Because it operates on the rendered visual model, it is immune to the shift from table-based layouts to modern CSS-based designs. However, the method has limitations:

  • ASCII Tables: It cannot extract tables created with plain text and spaces (e.g., inside <pre> tags) because there are no individual DOM nodes to reason about.
  • Expansion Order: Occasionally, expanding in the wrong direction first can cause the algorithm to hit a dead end before capturing the full table.
  • Visual Noise: Tables that are visually "messy" or use non-aligned nested structures can result in false negatives.

The authors suggest that future iterations could integrate more visual metadata, such as background colors and font styles, or apply machine learning to the spatial features to further improve robustness.

Summary updated

BibTeX
@inproceedings{gatterbauer2006table,
  author = {Wolfgang Gatterbauer and Paul Bohunsky},
  title = {{Table Extraction Using Spatial Reasoning on the CSS2 Visual Box Model}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2006},
}
Figure