Coordination of Emergent Demand Changes via Value-Based Negotiation for Supply Chain Management (Student Abstract)

Takumu Shimizu, Ryota Higa, Katsuhide Fujita, Shinji NakadaiAAAI 2024 · DOI

Introduction

Supply Chain Management (SCM) is inherently volatile, frequently disrupted by "emergent" demand changes—sudden spikes or drops that deviate from planned quantities. While Reinforcement Learning (RL) has shown promise in optimizing SCM, most existing research assumes a centralized environment where a single controller manages the entire chain. In reality, supply chains are composed of independent entities that must coordinate to reach agreements. Static policies often fail during sudden shifts because they lack a mechanism for decentralized, real-time plan adjustment.

The Proposed Framework

The authors introduce an automated negotiation framework that allows independent RL agents to adapt to sudden demand fluctuations. The model utilizes a four-layer serial supply chain consisting of a Terminal Supplier (TS), a Supplier (S), a Customer (C), and a Terminal Customer (TC).

Overview of the decentralized negotiation framework showing the interaction between RL agents and the plan adjustment process.

Key Innovations

  • Value-Based Utility: Instead of using hard-coded utility functions, agents use their RL value function ($V$) to estimate the "worth" of a proposed plan. By running internal simulations, an agent uses its learned experience to judge whether a new agreement is beneficial.
  • Negotiation Trigger: Agents initiate negotiation only when the difference between their current state's value and the potential value of a new plan exceeds a specific threshold ($\tau$). This allows for early detection of terminal demand changes before inventory shortages occur.
  • Alternating Offers Protocol (AOP): Agents trade offers back and forth to reach a consensus, allowing for iterative refinement of plans during complex, evolving crises.

Experimental Results

The researchers evaluated their PrePV-Nego framework against a "Zero-Shot" RL agent (no negotiation) and a standard "Base-Stock" heuristic across four demand scenarios. The primary performance metric was the Achievement Rate (AR), which measures the percentage of steps where inventory was sufficient to meet demand.

Comparison of Achievement Rates (AR) for Supplier (S) and Customer (C) agents across different demand scenarios.

As shown in the results table, PrePV-Nego achieved a 1.00 (100%) achievement rate across all scenarios, including the most challenging "Increase (large)" scenario. In contrast, the Zero-Shot agent's performance plummeted to 0.43 for the Customer agent, and the Base-Stock model dropped to 0.67. The negotiation process successfully "convinced" the chain to move more product, resulting in shipping quantities 33% higher and inventory levels 52% higher than the non-negotiating models.

Conclusion and Future Work

By leveraging the RL value function as a utility metric, this framework bridges the gap between long-term strategic planning and short-term tactical adjustment. While highly effective in a serial chain, future research is needed to address the computational costs of internal simulations and the scalability of negotiation overhead in more complex, non-linear supply networks. Additionally, exploring scenarios where agents have conflicting, rather than cooperative, incentives remains a key area for development.

Summary updated

BibTeX
@inproceedings{shimizu2024coordination,
  author = {Takumu Shimizu and Ryota Higa and Katsuhide Fujita and Shinji Nakadai},
  title = {{Coordination of Emergent Demand Changes via Value-Based Negotiation for Supply Chain Management (Student Abstract)}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i21.30510},
}
Figure