TL;DR
This paper tackles the challenge of automatically identifying online databases within the vast expanse of the web. They propose a novel hierarchical classification strategy, HIFI, that leverages both the structural and textual features of web forms to accurately distinguish database entry points from other web content, offering a scalable solution for information integration.
What Problem Does This Paper Address?
The internet is awash with information, much of which resides in online databases. However, finding these databases is a significant challenge. Traditional search engines return millions of documents, many of which are not databases themselves but rather articles, personal pages, or other irrelevant content. While curated directories of databases exist, they are difficult to maintain due to the sheer volume and constant evolution of online data. Existing automated methods for identifying database entry points (typically web forms) struggle with the inherent variability in form structure and content, and the fact that forms for different database domains can appear similar, while forms within the same domain can look quite different. This paper aims to bridge this gap by developing a robust and scalable method to automatically classify web forms, identifying those that serve as entry points to databases within a specific domain.
Methods
The core of the proposed solution is a hierarchical classification strategy called HIFI (Hierarchical Form Identification). HIFI breaks down the problem into two stages:
- Generic Form Classifier (GFC): This classifier focuses on the structural features of web forms (e.g., number of textboxes, selection lists, checkboxes, hidden fields). Its goal is to distinguish between "searchable" forms (potential database entry points) and "non-searchable" forms (like login or subscription forms). The authors found that decision trees (specifically C4.5) perform best for this structural classification. This stage is domain-independent, meaning the same GFC can be used across different database domains.
- Domain-Specific Form Classifier (DSFC): This classifier takes the output of the GFC (i.e., only the forms identified as searchable) and analyzes their textual content (e.g., form labels, text within input fields). Its purpose is to determine if a searchable form belongs to a specific target database domain. For this stage, Support Vector Machines (SVMs) with polynomial kernels proved most effective.
This hierarchical approach allows for simpler classifiers, as each classifier deals with a more constrained set of features and a more specific problem. It also enables the use of learning techniques best suited for each type of feature.
Datasets
The experiments were conducted using real web data collected by a focused crawler. The authors evaluated their approach on eight distinct database domains: Airfare, Auto, Book, Hotel, Job, Movie, Music, and Rental. For training and testing, they extracted structured forms and manually selected positive (relevant) and negative (irrelevant) examples. Specifically, they used approximately 220 positive and 220 negative examples per domain for the DSFC. The GFC was trained on a larger set of 216 searchable forms and 259 non-searchable forms.
Key Insights and Ideas
- Structure vs. Content is Key: The paper highlights that combining structural form features (for initial filtering of non-searchable forms) with textual content (for domain-specific classification) is crucial for high accuracy. Relying on only one type of feature is insufficient.
- Hierarchical Decomposition Simplifies: Breaking down the complex problem of database identification into two simpler, sequential classification tasks (searchable vs. non-searchable, then domain-specific) significantly improves performance and robustness.
- Domain Independence of GFC is Valuable: The GFC's ability to accurately identify searchable forms across various domains makes it a reusable component, reducing the need to build domain-specific classifiers for this initial filtering step.
- SVMs Excel at Text, Trees at Structure: The empirical results strongly support the choice of decision trees for structural features and SVMs for textual content, demonstrating that different machine learning algorithms are optimal for different data types.
- Crawler Quality Matters: The quality of the input forms (i.e., the percentage of relevant forms retrieved by the crawler) directly impacts the performance of the classification system. A more focused crawler yields better results.
Key Results
The HIFI strategy achieved high performance across the eight tested domains. Key results include:
- High Accuracy: The combined HIFI system (GFC + DSFC) achieved overall accuracy rates generally above 90% across most domains, with some reaching as high as 99%.
- Strong Recall and Precision: For most domains, HIFI demonstrated recall values ranging from 0.73 to 0.96 and precision values from 0.8 to 0.97. This indicates that the system effectively identifies relevant forms while minimizing false positives.
- GFC Effectiveness: The GFC showed high specificity (above 90%) in identifying searchable forms, meaning it effectively filtered out non-searchable forms. It also maintained high recall, misclassifying only a small fraction of relevant forms.
- HIFI Outperforms Monolithic Classifiers: When compared to monolithic classifiers (either using only DSFC or a combined feature set), HIFI consistently showed better performance, particularly in terms of recall.
Limitations and Future Work
- Simple Search Forms: The paper acknowledges that HIFI, relying on form content, may struggle to reliably identify the domain of very simple search forms that lack informative content.
- Crawler Dependency: The performance of HIFI is sensitive to the quality of the input forms provided by the crawler. A less focused crawler can lead to lower performance.
- Future Work: The authors propose combining their pre-query classification strategy with probing-based post-query methods to address the limitation with simple search forms. This would involve sending actual queries to forms and analyzing the results to further refine classification.
Summary updated