Introduction
Multi-objective task scheduling (MOTS) is a critical challenge in systems where performance must be balanced against accuracy, such as in malware detection or medical diagnostics. Traditional Deep Reinforcement Learning (DRL) approaches often struggle with reward complexity, queue position bias, and fixed input constraints. To address these issues, researchers have developed MERLIN, a hierarchical DRL framework designed for robust multi-objective scheduling across varying queue sizes.
The MERLIN Framework
MERLIN employs a two-tier hierarchical architecture to separate the concerns of task execution and queue management. This decoupling allows the system to handle conflicting goals like "accuracy" and "speed" more effectively than single-agent models.
Internal Agent ($p^i$)
The internal agent focuses on processing a single item. It determines the optimal sequence of sub-processes (e.g., specific malware detectors) to reach a conclusion while minimizing time. Crucially, once trained, the internal agent is frozen. This ensures "procedural fairness," where every item is treated with identical logic regardless of the system's current backlog or queue position.
Outer Agent ($p^o$)
The outer agent serves as the scheduler, observing the state of all items in the queue and selecting the next candidate for the internal agent. To handle queues larger than its training size, MERLIN utilizes a recursive "state-action reduction" process. It partitions large queues into manageable subsets, iteratively selecting the best candidates until the final scheduling decision is made. This allows a model trained on a queue of 10 items to successfully manage queues of 100+ items without retraining.
Experimental Evaluation
The framework was evaluated using a malware detection dataset of 25,000 executable files. This environment is characterized by high uncertainty, as the true processing time for any given file is unknown until analysis is complete. MERLIN was trained on small queues of 10 items and tested on significantly larger queues to assess its scaling capabilities.
Key Results
MERLIN demonstrated superior efficiency compared to standard heuristic baselines. By learning to infer task difficulty through interaction rather than relying on pre-known durations, the agents proved highly robust in real-world scenarios.
Key findings include:
- Efficiency Gains: MERLIN outperformed realistic baselines like First-Come-First-Serve (FCFS) and Smallest-File-First (SFF) by over 22% across all tested queue sizes.
- Reduced Latency: In fixed-size experiments, MERLIN achieved job completion times that were 27% to 71% shorter than traditional methods.
- Computational Scalability: Training on small queue sizes (n=10) and scaling up proved far more efficient than training large networks. A model trained for a 20-item queue took seven times longer to reach the same performance level as the 10-item MERLIN model.
Conclusion
MERLIN provides a scalable and fair solution for complex scheduling tasks. While the current implementation focuses on malware detection, its hierarchical structure is applicable to other domains like industrial maintenance or medical testing. Future work will explore the impact of setup costs in physical systems and investigate methods to further approach global optimality in extremely large-scale environments.
Summary updated