Abstract
Data-intensive applications, such as data mining and data visualization, need to deal with huge amount of data. Service-oriented architecture, on the other hand, offers a scalable and flexible framework to implement loosely-coupled and standards-based distribute computing which the data-intensive applications usually require. In this paper, we apply four kinds of data-flow constructs, namely Map Construct, Reduce Construct, Conditional Construct and Loop Construct, to assist Web services composition to process huge volume of data. Furthermore, we put forward the approach to the composition of Web services based on the I/O operation ratio, in which the CPU intensive Web services and the I/O intensive ones are arranged to execute in parallel as far as possible. A case is presented finally to show that our approaches are feasible and effective.
Introduction
In Service-Oriented Architecture (SOA), Web services provide a flexible framework for building loosely coupled applications. However, standard composition tools like BPEL are primarily designed for control-flow logic and often struggle with data-intensive workloads, such as large-scale data mining. These tasks involve massive datasets that can lead to resource contention and hardware bottlenecks when multiple services compete for the same resources. This paper addresses the gap between SOA flexibility and the efficiency required for parallel data processing by introducing specialized data-flow constructs and a novel scheduling algorithm.
Methodology
The proposed framework optimizes data-intensive Web services through two primary mechanisms:
1. Data-flow Constructs
The authors adapt four scientific workflow constructs to Web services to handle large data volumes efficiently:
- Map (M): Distributes data across multiple service replicas for parallel processing.
- Reduce (R): Aggregates results sequentially.
- Conditional (C): Executes services based on data predicates.
- Loop (L): Iteratively executes services based on previous outputs.
2. I/O Operation Ratio Scheduling
A novel scheduling algorithm utilizes the "I/O operation ratio"—a measure of a service's reliance on disk/network access versus CPU calculation. By identifying and pairing CPU-intensive services (low I/O ratio) with I/O-intensive services (high I/O ratio) that have no data dependencies, the system can interleave execution. This ensures that while one task is blocked by I/O, the other utilizes the CPU, maximizing hardware throughput.
Experimental Results
The approach was evaluated using a distributed data mining platform. The workflow included data preprocessing (Conditional), classification (Map and Reduce), and model presentation (Loop).
As shown in the performance analysis, increasing the number of nodes for the classification phase (using the Map construct) led to a significant reduction in total runtime. With a single node, the runtime was approximately 140 seconds, which dropped to roughly 40 seconds when scaled to 8 nodes. Notably, the time spent on model mining (Map) saw the most dramatic decrease, while preprocessing and presentation times remained relatively constant, demonstrating that the optimizations effectively targeted the primary bottlenecks.
Discussion and Conclusion
The study highlights that effective parallelism in SOA is not merely about increasing the number of tasks but about resource complementarity—pairing tasks with different resource needs to maximize utility. While the Map construct offers significant speedups, it introduces a trade-off in terms of the financial cost of maintaining multiple service replicas. Future work will focus on tighter integration of data-flow constructs with the I/O ratio scheduling engine and testing the framework across more diverse cloud environments with varying network latencies.
Summary updated