Abstract
Negotiation has been extensively discussed in game-theoretic, economic, and management science literatures for decades. Recent growing interest in electronic commerce has given increased importance to automated negotiation. Evidence both from theoretical analysis and from observations of human interactions suggests that if decision makers can somehow take into consideration what other agents are thinking and furthermore learn during their interactions how other agents behave, their payoff might increase. In this paper, we propose a sequential decision making model of negotiation, called Bazaar. Within the proposed negotiation framework, we model learning as a Bayesian belief update process. In this paper, we explore the hypothesis that learning is beneficial in sequential negotiation and present initial experimental results.
Introduction
As electronic commerce expanded in the mid-1990s, the demand for autonomous software agents capable of sophisticated negotiation grew. Traditional game-theoretic models often prioritized final outcomes over the negotiation process itself, frequently lacking the flexibility required for the dynamic "give-and-take" of real-world bargaining. These models typically failed to account for how an agent might adapt its strategy based on an opponent's behavior mid-negotiation. The Bazaar framework addresses this gap by treating negotiation as a sequential decision-making process where feedback is used to continuously update internal models of the environment and other participants.
The Bazaar Framework
The Bazaar model formalizes negotiation as a sequence of stages where each action depends on previous interactions, enabling robust feedback loops. The core of the framework is a Bayesian belief update process. Agents maintain a subjective probability distribution over a set of hypotheses regarding the opponent's hidden constraints, such as their reservation price (the maximum a buyer will pay or the minimum a seller will accept).
When a new offer is received, the agent applies Bayesian rules to update the probability of these hypotheses. In the current implementation, these beliefs are represented as piecewise probability distributions—specifically, vectors of 101 elements—allowing for a granular, non-parametric representation of the opponent's likely reservation price.
To determine the optimal next move, an agent performs a recursive evaluation to maximize its expected payoff. This calculation accounts for implementation costs (e.g., time and communication overhead) and predicts future responses from the opponent.
Experimental Evaluation
The Bazaar framework was evaluated using a simulated bargaining scenario consisting of 500 random negotiation instances. Supplier reservation prices were uniformly distributed between [0, 49], while buyer reservation prices ranged between [50, 100]. This configuration ensured that a "Zone of Agreement" (ZOPA)—a price range acceptable to both parties—always existed. Performance was measured by Joint Utility (with a theoretical Nash maximum of 0.25) and the number of proposals exchanged, which serves as a proxy for negotiation efficiency.
Key Insights and Results
The experimental results demonstrate that negotiation is effectively an information-gathering process. Every counter-offer serves as a signal revealing the opponent's hidden constraints.
The Advantage of Mutual Learning
The most efficient and mutually beneficial outcomes occurred when both agents employed learning strategies. This "Both Learn" configuration achieved a joint utility of 0.22, remarkably close to the theoretical Nash maximum, and reduced the number of rounds required to reach an agreement by nearly 30% compared to non-learning models.
| Configuration | Joint Utility (Avg) | # of Proposals (Avg) |
|---|---|---|
| Both Learn | 0.22 | 24 |
| Neither Learn | 0.18 | 34 |
| Only Buyer Learns | 0.15 | 28 |
Asymmetric Learning and Flexibility
Interestingly, asymmetric learning—where only one party learns—can be counterproductive for joint welfare. When only the buyer learned, joint utility dropped to 0.15. While the learning buyer improved their individual position, they exploited the non-learning supplier so effectively that the overall "win-win" quality of the deal suffered.
Furthermore, the benefits of learning are positively correlated with the size of the Zone of Agreement. As the ZOPA increases, agents have more room to maneuver, and the value of accurately modeling an opponent's behavior grows significantly.
Conclusion
The Bazaar framework illustrates that incorporating Bayesian learning into automated negotiation leads to faster agreements and higher-quality outcomes, particularly when both parties are capable of learning. While this study focused on single-issue price negotiations with linear utility functions, it provides a foundation for more complex scenarios. Future work will extend Bazaar to multi-issue negotiations and apply it to real-world domains such as supply chain management, where multi-criteria decision-making is essential.
Summary updated