Learning Semantic Descriptions of Web Information Sources

Mark James Carman, Craig A. KnoblockIJCAI 2007

Introduction

The proliferation of structured data on the web, accessible via Web Services and RSS feeds, presents a significant opportunity for automated data integration. However, programmatically utilizing this data—such as building an agent that synthesizes weather, flight, and hotel information—requires machines to understand the semantic meaning and functional relationships of the data fields. While existing techniques can identify basic semantic types (e.g., "zipcode"), they often fail to capture the complex logic behind the data (e.g., whether a temperature is a current reading or a forecast). This paper presents a framework for automatically generating semantic descriptions by learning Datalog definitions through active invocation and Inductive Logic Programming (ILP).

Methodology

The proposed system treats the discovery of source models as an active search problem. The process begins with Active Invocation, where the system queries a new source with a set of random inputs to observe its output.

Following invocation, the system performs an Inductive Search. This is a top-down, best-first search through a space of potential Datalog definitions. Starting from an empty clause, the system iteratively adds literals—references to known sources or domain relations—to construct increasingly complex models.

The top-down, best-first search algorithm used to discover Datalog definitions.

Candidate definitions are evaluated using Jaccard Similarity, comparing the results produced by the candidate model against the actual results from the web source. To maintain computational efficiency, the search space is pruned using standard ILP constraints, such as limiting definition length and ensuring executability. Furthermore, the system employs approximate matching techniques (e.g., Jaro-Winkler for text and fuzzy matching for numbers) to account for minor data variations across different sources.

Experimental Setup

The framework was evaluated using 25 real-world web services spanning five domains: Geospatial, Financial, Weather, Hotels, and Cars. The underlying knowledge base consisted of 70 semantic types and 36 relations derived from 35 previously modeled services. This setup allowed the learner to describe new sources as compositions of existing ones, effectively modeling by comparison.

Results and Analysis

The system successfully identified the intended semantic definitions for 19 out of the 25 test cases. On average, the framework achieved a precision of 88% and a recall of 69%.

Performance metrics across five distinct domains, showing precision and recall.

Performance varied by domain. The Geospatial domain saw perfect precision (100%), attributed to the mathematically precise nature of its relationships. In the Financial domain, the system demonstrated sophisticated reasoning, discovering that a "current price" could be modeled as the sum of the "previous day's close" and "today's change." The system also proved robust in handling partial information by penalizing missing attributes based on domain size rather than outright failure.

Conclusion and Future Work

While the framework effectively automates the generation of semantic descriptions for conjunctive queries, it currently faces limitations. It cannot yet model sources involving complex logic such as unions, negations, or aggregations. Future research will focus on scaling the system to thousands of sources, incorporating constants, and handling hierarchical semantic types. Despite these challenges, the results demonstrate that active learning and ILP provide a powerful foundation for making web data machine-understandable.

Summary updated

BibTeX
@inproceedings{carman2007learning,
  author = {Mark James Carman and Craig A. Knoblock},
  title = {{Learning Semantic Descriptions of Web Information Sources}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2007},
}
Figure