Answering Table Queries on the Web using Column Keywords

Rakesh Pimplikar, Sunita Sarawagi

Introduction

The web is home to an immense repository of structured data embedded within millions of HTML tables. However, accessing this data in a unified way is challenging because it lacks a central schema and often contains noise or missing metadata. The WWT (Web Window on Tables) system, presented at VLDB 2012, addresses this by enabling structured search across these disparate sources. Unlike traditional search engines that return a list of links, WWT synthesizes a single, consolidated answer table from multiple relevant web tables based on keyword-based column descriptions.

The primary difficulty in mining web tables lies in their "in the wild" nature. An estimated 80% of tables lack proper header tags, and many omit headers entirely or use cryptic labels. Furthermore, relevant data is often fragmented; one table might contain an explorer's name and nationality, while another lists the same explorer's areas of discovery.

Figure 1: The WWT task involves mapping query column descriptors (Q1, Q2, Q3) to heterogeneous web tables and merging the results into a single answer table.

To provide a coherent answer, a system must solve two core problems:

  1. Column Mapping: Determining which columns in various web tables correspond to the user's query keywords.
  2. Consolidation: Merging these fragments into a clean, deduplicated response.

The WWT System Architecture

WWT employs a sophisticated pipeline to transform raw web data into structured answers. The process begins with an offline phase where tables are extracted, and their headers and surrounding context are indexed to facilitate rapid retrieval.

Figure 2: Overview of the WWT system architecture, showing the flow from query building to table ranking and consolidation.

Two-Stage Indexing and Joint Labeling

WWT uses a novel two-stage indexing strategy. First, it searches for keywords in headers and context. Second, it performs a content-based probe: if a table is found to be highly relevant, the system looks for other tables sharing its cell values, even if their metadata is poor. This strategy was highly effective, with 50% of all relevant source tables discovered only during the second stage.

The heart of the system is a probabilistic graphical model that treats column mapping as a joint labeling problem. Instead of labeling each table in isolation, WWT considers all candidate tables simultaneously. This allows for "label transfer"—if one table has a clear "Explorer Name" header and another has similar data under a generic "Name" header, the model can use the content overlap to correctly label the latter.

Key Results and Insights

The WWT system demonstrates significant improvements over baseline information retrieval (IR) methods:

  • Accuracy: The joint inference model achieved an F1 score of 70%, compared to 65% for standard IR.
  • Efficiency: Despite the complexity of the graphical model, the system maintains an average query time of roughly 6.7 seconds, making it viable for near-real-time search.

One of the most important findings was that context is king. Query terms are frequently split between a table's header and the text surrounding it on the webpage. By dynamically segmenting queries between these two sources using the SegSim model, WWT reduced mapping errors by over 10%.

Conclusion

WWT represents a significant step forward in structured web search. By moving beyond simple table ranking to joint column mapping and consolidation, it unlocks the latent value of millions of web tables. Future work aims to integrate formal ontologies to further improve labeling accuracy in cases where web data is sparse or ambiguous.

Summary updated

BibTeX
@article{pimplikar2012answering,
  author = {Rakesh Pimplikar and Sunita Sarawagi},
  title = {{Answering Table Queries on the Web using Column Keywords}},
  journal = {Proc. VLDB Endow.},
  year = {2012},
  url = {https://openreview.net/forum?id=V9jJDknPJf0},
}
Figure