Abstract
This paper introduces a novel approach to warehouse logistics by integrating dock mode decisions—choosing whether a dock handles loading, unloading, or both—directly into truck scheduling. To solve this complex problem, the authors developed Q-ALNS, an algorithm that embeds reinforcement learning (Q-learning) into an Adaptive Large Neighborhood Search (ALNS) to intelligently navigate the massive search space of possible schedules. Experimental results show that this integrated approach significantly reduces tardiness and makespan compared to traditional fixed-mode dock strategies.
Problem Definition: TASP-DMD
In traditional warehouse management, docks are often "hard-coded" as either loading-only or unloading-only. While "Mixed Service Mode" (MSM) docks—which can do both—exist, most research treats the number and location of these docks as fixed inputs decided well before daily operations begin.
Real-world demand is volatile; inbound and outbound freight volumes fluctuate daily. This paper addresses the Integrated Trucks Assignment and Scheduling Problem with Dock Mode Decision (TASP-DMD). The objective is to simultaneously optimize:
- Dock Mode: Dynamically deciding which docks should be loading, unloading, or mixed.
- Assignment: Mapping specific trucks to specific docks.
- Scheduling: Determining the processing order to minimize tardiness, total makespan, and internal transport distance for Automated Guided Vehicles (AGVs).
As shown in the literature comparison below, this research is unique in its simultaneous consideration of MSM, internal handling, and dynamic mode variables.
Methodology: The Q-ALNS Algorithm
The core of the solution is Q-ALNS, a hybrid meta-heuristic algorithm designed for efficiency and adaptability.
- Two-Layer Search: The algorithm employs an outer loop to perturb dock mode decisions and an inner loop to optimize truck sequences using "destroy and repair" operators.
- Q-Learning for Operator Selection: Unlike standard ALNS, which uses simple score-based selection, Q-ALNS utilizes a Q-learning agent. The agent treats the search process as an environment, learning which combinations of operators yield the best long-term rewards through an epsilon-greedy strategy that balances exploration and exploitation.
- Operator Filtering (OF): To boost efficiency, the authors used a Wilcoxon dominance matrix to prune 16 potential operators down to the 3 most effective ones, ensuring a lean and high-performing search process.
Experimental Setup and Results
The study utilized data from a real-world unmanned warehouse in Linyi City, China, generating 10 sets of test instances ranging from 20 to 200 trucks. Q-ALNS was benchmarked against Gurobi (for small instances), traditional S-ALNS, and Random Large Neighborhood Search (RLNS).
Key Findings
- Superior Efficiency: Q-ALNS consistently outperformed benchmarks in finding the Pareto front (the optimal trade-off between tardiness, makespan, and distance). It improved average tardiness by 9.0% to 9.4% over traditional ALNS variants.
- Adaptability Gains: The adaptive dock strategy achieved high utilization (nearly 90%) while using fewer mixed-mode docks than a pure "Mix" strategy. This resulted in a 2.2% to 11.7% reduction in tardiness compared to fixed-mode arrangements.
- Scalability: While Q-ALNS has a slight computational overhead for small problems, its efficiency relative to competitors increases as the problem size grows, making it suitable for large-scale distribution centers.
Conclusion
By treating dock modes as a dynamic decision variable rather than a static constraint, this research provides a more flexible framework for modern logistics. The integration of Q-learning allows the algorithm to avoid local optima and adapt to complex search landscapes. Future work will focus on incorporating stochastic elements, such as uncertain truck arrival times and AGV reliability, to further bridge the gap between simulation and real-world operations.
Summary updated