A Process-Aware Security Task Scheduling Algorithm

Tao Zhang, Yuanyuan Ma, Cheng Zhou, Tao LiJournal of Digital Information Management 2014

Abstract

In this paper, a process-aware security task-scheduling algorithm is proposed as IOAware. The algorithm evaluates the performance of hardware in computing nodes and estimates properties of a task while it is being executed. In subsequent assignments, the algorithm assigns tasks based on the performance of the TaskTracker and the task properties. To verify the theoretical feasibility of the proposed IOAware scheduling algorithm, a schedule model is proposed to implement the method. After applying the scheduling module in a Hadoop cluster with multiple experiments, the results show that, compared with the performances of the FIFO, computing capacity-scheduling, and fair-scheduling algorithms, the IOAware scheduling algorithm effectively reduces the execution time of tasks and improves the throughput of the cluster.

The Problem: Heterogeneity and IO Bottlenecks

In large-scale distributed systems like Hadoop, performance often suffers because standard schedulers (like FIFO, Capacity, or Fair) treat computing nodes as relatively uniform. In reality, clusters are often heterogeneous, consisting of hardware with varying CPU speeds and disk performance.

When multiple IO-intensive tasks are assigned to the same node, they compete for disk bandwidth, leaving the CPU idle while waiting for data—a classic IO bottleneck. Conversely, CPU-heavy tasks might leave disk resources underutilized. The IOAware scheduler aims to bridge this gap by understanding the specific resource requirements of a task and matching them to the available hardware.

The IOAware Methodology

The IOAware framework operates in three main phases:

  1. Hardware Assessment: The algorithm quantitatively estimates the disk performance ($R_{disk}$) of each node by analyzing CPU frequency and execution times of sample tasks.
  2. Job Classification: Jobs are classified as either CPU-bound or IO-bound. This is determined by calculating the ratio between Map Input Data (MID) and Map Output Data (MOD).
  3. Dynamic Task Dispatching: The scheduler uses a specific logic flow to assign tasks. If a node is struggling with blocked IO tasks, the scheduler will intentionally assign it a CPU-bound task to maximize resource utilization without further choking the disk.
Logic flow for dynamic task dispatching in the IOAware algorithm.

Experimental Results

The researchers evaluated IOAware using a heterogeneous Hadoop cluster running three standard benchmarks: WordCount (CPU-bound), Grep (CPU-bound), and TeraSort (IO-bound). Through experimentation, the authors identified a threshold value of 0.6 for judging task "obstruction," which provided the optimal balance for minimizing average response time.

When compared against standard scheduling algorithms, IOAware demonstrated superior efficiency. Specifically, it completed the test suite in 1115 seconds, outperforming the FIFO (1123s), Fair (1125s), and Capacity (1135s) schedulers.

Total execution time comparison between IOAware and standard Hadoop schedulers.

Beyond execution time, IOAware showed improved data localization rates and more stable CPU utilization. Visualization of system resources indicated that IOAware produced smoother utilization curves with fewer extreme spikes, suggesting a more predictable and stable cluster environment.

Conclusion

The IOAware algorithm effectively optimizes Hadoop clusters by dynamically matching task resource needs with hardware capabilities. By intelligently co-locating complementary tasks, it reduces disk bottlenecks and increases overall system throughput. Future work may involve extending this process-aware logic to modern architectures like ARM or GPU computing, which are increasingly prevalent in contemporary data centers.

Summary updated

Abstract

In this paper, a process-aware security task- scheduling algorithm is proposed as IOAware. The algorithm evaluates the performance of hardware in computing nodes, and estimates properties of a task while it is being executed. In subsequent assignments, the algorithm assigns tasks based on the performance of the TaskTracker and the task properties. To verify the theoretical feasibility of the proposed IOAware scheduling algorithm, a schedule model is proposed to implement the method. After applying the scheduling module in a Hadoop cluster with multiple experiments, the results show that, compared with the performances of the FIFO, computing capacity-scheduling, and fair-scheduling algorithms in terms of four aspects: response time, localization ratio of the data, system throughput, and system resources, IOAware scheduling algorithm can reach up to the disk IO effect of the Shared computing nodes, effectively reduce the execution time of the tasks and improve the throughput of the cluster.
BibTeX
@article{zhang2014process,
  author = {Tao Zhang and Yuanyuan Ma and Cheng Zhou and Tao Li},
  title = {{A Process-Aware Security Task Scheduling Algorithm}},
  journal = {Journal of Digital Information Management},
  year = {2014},
}
Figure