Abstract
Predicting port congestion is crucial for maintaining reliable global supply chains. Accurate forecasts enable improved shipment planning, reduce delays and costs, and optimize inventory and distribution strategies, thereby ensuring timely deliveries and enhancing supply chain resilience. To achieve accurate predictions, analyzing vessel behavior and their stay times at specific port terminals is essential, focusing particularly on berth scheduling under various conditions. Crucially, the model must capture and learn the underlying priorities and patterns of berth scheduling. Berth scheduling and planning are influenced by a range of factors, including incoming vessel size, waiting times, and the status of vessels within the port terminal. By observing historical Automatic Identification System (AIS) positions of vessels, we reconstruct berth schedules, which are subsequently utilized to determine the reward function via Inverse Reinforcement Learning (IRL). For this purpose, we modeled a specific terminal at the Port of New York/New Jersey and developed Temporal-IRL. This Temporal-IRL model learns berth scheduling to predict vessel sequencing at the terminal and estimate vessel port stay, encompassing both waiting and berthing times, to forecast port congestion. Utilizing data from Maher Terminal spanning January 2015 to September 2023, we trained and tested the model, achieving demonstrably excellent results.
The Challenge of Port Congestion
Global supply chains rely heavily on efficient port operations, but predicting port congestion remains a massive challenge. When ships arrive at a port, they aren't simply served on a first-come, first-served basis. Port operators juggle a complex mix of variables—vessel size, carrier company, wait times, channel depth, and contractual priorities—to assign berths.
Traditional machine learning models struggle to capture these tacit, intricate operational rules. Conversely, while Reinforcement Learning (RL) is an excellent framework for decision-making, manually engineering a reward function that accurately reflects all these competing real-world priorities is practically impossible. The authors needed a way to learn the port's underlying scheduling logic directly from historical data without explicitly programming the rules themselves.
Methodology: The Temporal-IRL Pipeline
The authors propose Temporal-IRL, a pipeline that treats berth scheduling as an inverse problem. Instead of guessing the rules, they use Inverse Reinforcement Learning (IRL) to observe "expert behavior" (historical ship movements) and infer the hidden reward function guiding those decisions.
- State & Action Modeling: The port is modeled as 19 discrete slots (6 berths, 7 waiting areas, 6 incoming slots). Continuous time is discretized into 8-hour windows based on average vessel stay durations.
- Temporal Feature Extraction: Raw vessel data (size, carrier, stay time) is fed into an LSTM-Autoencoder (LSTM-AE). The LSTM captures the sequential, time-series nature of the data, while the Autoencoder compresses this into dense feature representations.
- Maximum Entropy IRL: These temporal features are passed into a MaxEnt IRL algorithm. By maximizing the entropy of the policy while matching the expected feature counts of the historical data, the model learns a reward function that best explains the port operators' historical scheduling decisions.
Data and Action Modeling
The study utilizes historical Automatic Identification System (AIS) data from the Maher Terminal at the Port of New York and New Jersey, spanning January 2015 to September 2023. The dataset tracks vessel trajectories, arrival/departure times, waiting area durations, and specific berth assignments. Additional metadata, such as vessel size and operating company (carrier), was linked using unique IMO (International Maritime Organization) numbers.
Key Insights and Results
- Learning Priorities, Not Just Outcomes: By using IRL to infer the reward function, the model learns the priorities of the port operators (e.g., prioritizing larger ships or specific carriers). This makes the model much more robust than standard predictive ML, which just looks for statistical correlations.
- Upstream Commands Forecast Downstream Congestion: Instead of treating congestion as an isolated variable, the authors predict the berth scheduling instructions issued to vessels. Simulating these upstream commands naturally yields highly accurate downstream congestion forecasts.
- Action Accuracy: Temporal-IRL achieved an impressive 82.64% accuracy in predicting specific scheduling actions, significantly outperforming baseline models like standard LSTMs (70.13%) and Feedforward Neural Networks (55.83%).
- Vessel Departure Prediction: The model achieved state-of-the-art performance in predicting when a vessel would leave the system, hitting 71.51% accuracy.
- Congestion Prediction: For forecasting actual port congestion, Temporal-IRL reached 74.06% accuracy, proving its viability as a comprehensive scheduling and congestion forecasting tool.
Limitations and Future Work
The current model was evaluated on a single terminal and relies on a fixed 8-hour time discretization. Future work aims to incorporate external factors like weather conditions and refine the time-window approach to capture finer-grained actions. Expanding the framework to model throughput behaviors across entire regional supply chains remains a key objective for enhancing global logistics resilience.
Summary updated