ReflecSched: Solving Dynamic Flexible Job-Shop Scheduling via LLM-Powered Hierarchical Reflection

Shijie Cao, Yuan Yuan

Abstract

The NP-hard Dynamic Flexible Job-Shop Scheduling (DFJSP) problem involves real-time events and complex routing. While traditional rules are efficient but rigid, deep learning is opaque and requires feature engineering. Large Language Models (LLMs) promise adaptive reasoning without this engineering overhead, yet we find their direct application is suboptimal. Baseline LLMs suffer from three key pitfalls: the long-context paradox, where crucial data is underutilized; an underutilization of expert heuristics; and myopic decision-making. To address this, we propose ReflecSched, a framework that empowers the LLM beyond a direct scheduler by equipping it with a strategic analysis capability. ReflecSched tasks the LLM to analyze heuristic-driven simulations across multiple planning horizons and distill them into a concise, natural-language summary termed "Strategic Experience". This summary is then integrated into the prompt of a final decision-making module, guiding it to produce non-myopic actions. Experiments demonstrate ReflecSched achieves superior performance, with its best variants attaining an average RPD of 6.04% and rank of 3.18, significantly outperforming strong traditional and learning-based methods. It also statistically and decisively surpasses direct LLM baselines, securing a 71.35% Win Rate while being, on average, 15.1% more token-efficient on Normal-scale problems.

Introduction

Dynamic Flexible Job-Shop Scheduling (DFJSP) is a critical challenge in modern manufacturing, where machines can fail and new orders arrive unpredictably. The goal is to minimize makespan—the total time required to complete all jobs. Traditional approaches typically rely on simple heuristics, which are fast but rigid, or Deep Reinforcement Learning (DRL), which requires extensive training and complex feature engineering.

While Large Language Models (LLMs) offer a promising alternative due to their reasoning capabilities, direct application reveals significant limitations. The "Long-Context Paradox" causes LLMs to ignore static data as context grows, while "Myopic Greed" leads them to prioritize immediate task completion over long-term bottleneck prevention. Furthermore, LLMs often struggle to consistently apply expert heuristics.

The ReflecSched Framework

ReflecSched addresses these pitfalls through a two-stage "Thinker-Actor" architecture. Instead of acting as a direct scheduler, the system uses a Hierarchical Reflection Module (the "Thinker"). This module runs multiple simulations using various heuristics to identify "Best" and "Worst" paths. The LLM then performs contrastive reasoning to understand the underlying causes of success or failure, distilling these insights into a natural-language "Strategic Experience".

Analysis of prompt length reduction and performance ratio across different problem scales.

This strategic summary acts as a form of lossy compression. By turning thousands of lines of simulation data into actionable advice (e.g., "Prioritize Machine 3 to avoid downstream bottlenecks"), ReflecSched overcomes the long-context paradox. As shown in the analysis above, this approach significantly reduces the token overhead associated with static data while maintaining high performance across both normal and small-scale problems.

Experimental Evaluation

The framework was tested across several benchmarks, including GEN-Bench, PDR-Bench, and MK-Bench. Notably, it was evaluated on JMS-Bench, which simulates a high-fidelity semiconductor cluster tool. This environment features complex re-entrant flows where wafers must visit the same process modules multiple times, creating intricate scheduling constraints.

Schematic of a semiconductor cluster tool used in the JMS-Bench evaluation.

Key Results

  • Performance: ReflecSched achieved an average RPD of 6.04%, significantly beating strong DRL baselines like HMP-SAC (11.00%) and IDDQN (10.45%).
  • Efficiency: The model was 15.1% more token-efficient than direct LLM baselines on normal-scale problems due to the strategic compression of context.
  • Robustness: In the complex semiconductor scenario (JMS-Bench), it maintained a dominant lead, proving it can handle highly technical, multi-stage constraints.
  • Win Rate: It secured a 71.35% win rate against direct LLM approaches, demonstrating that "thinking before acting" provides a decisive advantage.

Conclusion

ReflecSched demonstrates that LLMs are most effective in complex scheduling when used as strategic analysts rather than reactive controllers. By leveraging contrastive reasoning and strategic compression, the framework achieves state-of-the-art performance with zero-shot deployment. Future work will focus on reducing inference latency and exploring more diverse simulation strategies to further enhance the "Thinker" module's foresight.

Summary updated

Abstract

The NP-hard Dynamic Flexible Job-Shop Scheduling (DFJSP) problem involves real-time events and complex routing. While traditional rules are efficient but rigid, deep learning is opaque and requires feature engineering. Large Language Models (LLMs) promise adaptive reasoning without this engineering overhead, yet we find their direct application is suboptimal. Baseline LLMs suffer from three key pitfalls: the long-context paradox, where crucial data is underutilized; an underutilization of expert heuristics; and myopic decision-making. To address this, we propose ReflecSched, a framework that empowers the LLM beyond a direct scheduler by equipping it with a strategic analysis capability. ReflecSched tasks the LLM to analyze heuristic-driven simulations across multiple planning horizons and distill them into a concise, natural-language summary termed ``Strategic Experience''. This summary is then integrated into the prompt of a final decision-making module, guiding it to produce non-myopic actions. Experiments demonstrate ReflecSched achieves superior performance, with its best variants attaining an average RPD of 6.04\% and rank of 3.18, significantly outperforming strong traditional and learning-based methods. It also statistically and decisively surpasses direct LLM baselines, securing a 71.35\% Win Rate while being, on average, 15.1\% more token-efficient on Normal-scale problems. Ablation studies attribute this performance to a robust reflection mechanism that leverages high-quality, contrastive experience. This mechanism mitigates key LLM pitfalls like myopic greed, enabling ReflecSched to outperform all evaluated heuristics. Ultimately, the framework's performance is statistically on par with an oracle-like strategy, showcasing its effectiveness and robustness.
BibTeX
@misc{cao2025reflecsched,
  author = {Shijie Cao and Yuan Yuan},
  title = {{ReflecSched: Solving Dynamic Flexible Job-Shop Scheduling via LLM-Powered Hierarchical Reflection}},
  year = {2025},
  eprint = {2508.01724},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2508.01724},
}

Loading related content...

Figure