Temporal-IRL: Modeling Port Congestion and Berth Scheduling with Inverse Reinforcement Learning

Li Guo, Zixiang Xu, Wei Zhang, Yongli Hu, Xinyu Yang, Nikolay Aristov, Mingjie Tang, Elenna DugundjiarXiv 2025 · DOI · arXiv

Abstract

Predicting port congestion is crucial for maintaining reliable global supply chains. Accurate forecasts enable improved shipment planning, reduce delays and costs, and optimize inventory and distribution strategies, thereby ensuring timely deliveries and enhancing supply chain resilience. To achieve accurate predictions, analyzing vessel behavior and their stay times at specific port terminals is essential, focusing particularly on berth scheduling under various conditions. Crucially, the model must capture and learn the underlying priorities and patterns of berth scheduling. Berth scheduling and planning are influenced by a range of factors, including incoming vessel size, waiting times, and the status of vessels within the port terminal. By observing historical Automatic Identification System (AIS) positions of vessels, we reconstruct berth schedules, which are subsequently utilized to determine the reward function via Inverse Reinforcement Learning (IRL). For this purpose, we modeled a specific terminal at the Port of New York/New Jersey and developed Temporal-IRL. This Temporal-IRL model learns berth scheduling to predict vessel sequencing at the terminal and estimate vessel port stay, encompassing both waiting and berthing times, to forecast port congestion. Utilizing data from Maher Terminal spanning January 2015 to September 2023, we trained and tested the model, achieving demonstrably excellent results.

The Challenge of Port Congestion

Global supply chains rely heavily on efficient port operations, but predicting port congestion remains a massive challenge. When ships arrive at a port, they aren't simply served on a first-come, first-served basis. Port operators juggle a complex mix of variables—vessel size, carrier company, wait times, channel depth, and contractual priorities—to assign berths.

Traditional machine learning models struggle to capture these tacit, intricate operational rules. Conversely, while Reinforcement Learning (RL) is an excellent framework for decision-making, manually engineering a reward function that accurately reflects all these competing real-world priorities is practically impossible. The authors needed a way to learn the port's underlying scheduling logic directly from historical data without explicitly programming the rules themselves.

Methodology: The Temporal-IRL Pipeline

The authors propose Temporal-IRL, a pipeline that treats berth scheduling as an inverse problem. Instead of guessing the rules, they use Inverse Reinforcement Learning (IRL) to observe "expert behavior" (historical ship movements) and infer the hidden reward function guiding those decisions.

Overview of the Temporal-IRL pipeline, from raw AIS data to congestion prediction.
  1. State & Action Modeling: The port is modeled as 19 discrete slots (6 berths, 7 waiting areas, 6 incoming slots). Continuous time is discretized into 8-hour windows based on average vessel stay durations.
  2. Temporal Feature Extraction: Raw vessel data (size, carrier, stay time) is fed into an LSTM-Autoencoder (LSTM-AE). The LSTM captures the sequential, time-series nature of the data, while the Autoencoder compresses this into dense feature representations.
  3. Maximum Entropy IRL: These temporal features are passed into a MaxEnt IRL algorithm. By maximizing the entropy of the policy while matching the expected feature counts of the historical data, the model learns a reward function that best explains the port operators' historical scheduling decisions.

Data and Action Modeling

The study utilizes historical Automatic Identification System (AIS) data from the Maher Terminal at the Port of New York and New Jersey, spanning January 2015 to September 2023. The dataset tracks vessel trajectories, arrival/departure times, waiting area durations, and specific berth assignments. Additional metadata, such as vessel size and operating company (carrier), was linked using unique IMO (International Maritime Organization) numbers.

Distribution of vessel action types within the Maher Terminal dataset, illustrating the variety of scheduling decisions.

Key Insights and Results

  • Learning Priorities, Not Just Outcomes: By using IRL to infer the reward function, the model learns the priorities of the port operators (e.g., prioritizing larger ships or specific carriers). This makes the model much more robust than standard predictive ML, which just looks for statistical correlations.
  • Upstream Commands Forecast Downstream Congestion: Instead of treating congestion as an isolated variable, the authors predict the berth scheduling instructions issued to vessels. Simulating these upstream commands naturally yields highly accurate downstream congestion forecasts.
  • Action Accuracy: Temporal-IRL achieved an impressive 82.64% accuracy in predicting specific scheduling actions, significantly outperforming baseline models like standard LSTMs (70.13%) and Feedforward Neural Networks (55.83%).
  • Vessel Departure Prediction: The model achieved state-of-the-art performance in predicting when a vessel would leave the system, hitting 71.51% accuracy.
  • Congestion Prediction: For forecasting actual port congestion, Temporal-IRL reached 74.06% accuracy, proving its viability as a comprehensive scheduling and congestion forecasting tool.

Limitations and Future Work

The current model was evaluated on a single terminal and relies on a fixed 8-hour time discretization. Future work aims to incorporate external factors like weather conditions and refine the time-window approach to capture finer-grained actions. Expanding the framework to model throughput behaviors across entire regional supply chains remains a key objective for enhancing global logistics resilience.

Summary updated

Abstract

Predicting port congestion is crucial for maintaining reliable global supply chains. Accurate forecasts enableimprovedshipment planning, reducedelaysand costs, and optimizeinventoryanddistributionstrategies, thereby ensuring timely deliveries and enhancing supply chain resilience. To achieve accurate predictions, analyzing vessel behavior and their stay times at specific port terminals is essential, focusing particularly on berth scheduling under various conditions. Crucially, the model must capture and learn the underlying priorities and patterns of berth scheduling. Berth scheduling and planning are influenced by a range of factors, including incoming vessel size, waiting times, and the status of vessels within the port terminal. By observing historical Automatic Identification System (AIS) positions of vessels, we reconstruct berth schedules, which are subsequently utilized to determine the reward function via Inverse Reinforcement Learning (IRL). For this purpose, we modeled a specific terminal at the Port of New York/New Jersey and developed Temporal-IRL. This Temporal-IRL model learns berth scheduling to predict vessel sequencing at the terminal and estimate vessel port stay, encompassing both waiting and berthing times, to forecast port congestion. Utilizing data from Maher Terminal spanning January 2015 to September 2023, we trained and tested the model, achieving demonstrably excellent results.
BibTeX
@misc{guo2025temporal,
  author = {Li Guo and Zixiang Xu and Wei Zhang and Yongli Hu and Xinyu Yang and Nikolay Aristov and Mingjie Tang and Elenna Dugundji},
  title = {{Temporal-IRL: Modeling Port Congestion and Berth Scheduling with Inverse Reinforcement Learning}},
  year = {2025},
  eprint = {2506.19843},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2506.19843},
  howpublished = {arXiv},
}
Figure