Automatically Labeling the Inputs and Outputs of Web Services

Kristina Lerman, Anon Plangprasopchok, Craig A. KnoblockAAAI 2006

Abstract

The proliferation of Web services has created a significant challenge for automated data integration due to the "semantic gap"—the lack of machine-understandable meaning in service definitions. This paper presents a dual-strategy approach that combines metadata analysis of Web Service Definition Language (WSDL) files with active content testing. By invoking services and analyzing output patterns, the system achieves up to 80% accuracy in semantic labeling. This automated mapping is a critical step toward scaling information mediators that can autonomously discover and utilize diverse Web data sources.

Introduction

As the ecosystem of Web services expands, the ability to automatically combine data from multiple sources to answer complex queries remains a primary goal of information integration. However, the current standard for service description, WSDL, is purely syntactic. It defines technical parameters (e.g., data types like strings or integers) but fails to convey the semantic meaning of the data (e.g., whether a string represents a Zip Code or a City name). Traditionally, bridging this gap required manual mapping by human experts to a common ontology. To overcome this bottleneck, this research proposes an automated pipeline to discover the "meaning" of service inputs and outputs.

Methodology

The proposed system utilizes a two-stage classification pipeline:

  1. Metadata-based Classification (Inputs): The system extracts and cleans terms from WSDL files, including operation, message, and parameter names. Using Logistic Regression, it predicts the semantic type of inputs. For instance, a parameter named GetWeatherByZipRequest is decomposed into "weather" and "zip" to identify a "ZipCode" input type.
  2. Content-based Classification (Outputs): After identifying potential input types, the system performs "active testing." It retrieves sample data for the predicted input from a local database, invokes the live Web service, and captures the output. A domain-independent pattern language then analyzes the structure of the returned data (e.g., identifying 5-digit sequences as Zip Codes).

The core innovation lies in the Combined Approach. When the metadata classifier is uncertain, the system relies on content-based patterns. Conversely, if the output data is ambiguous (e.g., a number that could be temperature or humidity), the metadata provides the necessary context for disambiguation.

Experimental Results

The approach was evaluated using a corpus of 313 WSDL files and live validation on 13 operations from commercial Web services.

Performance Across Domains

The system demonstrated robust performance across various domains, including Geospatial, Weather, Airlines, and Electronics. The Logistic Regression model for input labeling achieved 87% accuracy, significantly outperforming Naive Bayes (64%), likely due to the high correlation between terms in service definitions.

F-measure performance across various service domains (Directory, Geospatial, Cars, Weather) comparing baseline (F0) and the proposed approach (F3).

For output labeling, the combined approach reached 80% accuracy, a substantial improvement over metadata-only (68%) or content-only (50%) methods. The effectiveness of the refined classification strategy is further evidenced in specific domains like Airlines and Electronics.

Comparison of F-measure for Airlines and Electronics domains under different experimental conditions (a) and (b), showing the advantage of the refined classification strategy.

Key Insights

  • Active Testing: Running the service acts as a vital validation step for input hypotheses.
  • Pattern Recognition: Analyzing the "shape" of data allows the system to recognize types even when developers use non-descriptive parameter names.
  • Real-world Viability: The system successfully invoked 12 out of 13 live operations, proving the feasibility of the "guess-and-test" loop in real-time environments.

Conclusion

This research demonstrates that combining metadata analysis with active content testing can effectively automate the semantic labeling of Web services. While the system shows high accuracy, challenges remain regarding sample size sensitivity in live services and handling complex mandatory input dependencies. Future work will focus on improving robustness through larger-scale data collection and enhancing the pattern-learning algorithm to handle diverse data formats.

Summary updated

BibTeX
@inproceedings{lerman2006automatically,
  author = {Kristina Lerman and Anon Plangprasopchok and Craig A. Knoblock},
  title = {{Automatically Labeling the Inputs and Outputs of Web Services}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2006},
}
Figure