TL;DR
This paper introduces the "Price Negotiator," a novel AI agent for online negotiation that leverages item images and external online listings to predict a fair agreement price. By disentangling price from language generation and incorporating a price-aware action prediction, it achieves more human-like and consistent negotiation outcomes compared to previous methods.
What Problem Does This Paper Address?
Negotiating a fair price for items in online marketplaces is a complex task that has proven challenging for AI agents. Existing dialogue systems often treat prices as just another word, leading to inconsistent pricing strategies and a failure to fully utilize all available information. This paper tackles the problem of building an intelligent agent that can effectively negotiate prices by considering not only the textual description but also the visual appearance of an item and by proactively estimating its true market value. The goal is to create an agent that can reach agreements that are both linguistically natural and economically sound, mimicking human negotiation behavior.
Methods
The Price Negotiator is a modular deep neural network composed of five key components:
- Online Value Estimator (OVE): This module uses a matching network to find similar items from an external online source. By analyzing the visual and textual features of these similar items, it predicts a potential market value for the item being negotiated. This is a novel approach that simulates how humans research prices before making a purchase.
- Hierarchical Recurrent Negotiation Encoder (HRNE): This component encodes the dialogue history and negotiation context. Crucially, it replaces explicit price values in utterances with a special token, treating them separately from general language generation. This helps to disentangle the numerical aspect of pricing from the linguistic aspect.
- Action Predictor Controller: Based on the dialogue state (including the estimated value from OVE), this module predicts the agent's next action, such as "negotiate," "concede," "offer," "accept," "reject," or "quit."
- Price Adjuster: When the action predictor decides to make a price adjustment, this module determines the specific price change ratio.
- Language Decoder: This module generates the natural language response based on the predicted action and dialogue state, ensuring that the agent's intentions are communicated effectively. A copy mechanism is used to integrate the predicted price into the generated utterance.
The model is trained using a combination of supervised learning and reinforcement learning (RL). The RL component, specifically the REINFORCE algorithm, is used to fine-tune the action predictor and price adjuster, encouraging more human-like negotiation strategies.
Datasets
The primary dataset used for training and evaluation is CraigslistBargain [2]. This dataset consists of human-generated negotiation dialogues over Craigslist advertisements. The paper also mentions simulating an online external data source by scraping 10,586 items from Craigslist to aid the Online Value Estimator.
Key Insights and Ideas
- Disentangling Price and Language: Treating prices as separate entities from general language generation is crucial for achieving consistent and human-like pricing strategies. This prevents the model from getting "confused" by numerical values within the language model.
- Online Value Estimation is Key: Proactively estimating an item's market value by looking at similar listings online is a powerful strategy. This "look-ahead" capability allows the agent to understand the item's true worth and negotiate more effectively.
- Modular Design for Negotiation: A modular approach, where different components handle specific aspects of negotiation (value estimation, dialogue encoding, action prediction, price adjustment, language generation), leads to better performance than end-to-end models.
- Reinforcement Learning for Human-like Behavior: Fine-tuning with RL significantly improves the agent's ability to adopt human-like negotiation tactics, leading to better agreement prices and more persuasive dialogues.
Key Results
The Price Negotiator model significantly outperforms baseline models on the CraigslistBargain dataset. Key results include:
- Improved Agreement Price: The model achieves agreement prices that are closer to human-agreed prices, with a notable reduction in human divergence (e.g., $125 for OVE+HRNE+AP+PA+LD vs. $325 for HRED).
- Enhanced Dialogue Quality: Metrics like IBLEU (Intent-BLEU) and sentence/vocabulary diversity show that the generated dialogues are more linguistically rich and human-like.
- Consistent Pricing: The model demonstrates significantly lower price inconsistency and offering inconsistency compared to baselines.
- Reinforcement Learning Benefits: Using RL further improves performance, especially in insisting on the OVE-estimated value and achieving better agreement prices.
- Human Studies: Human evaluations confirm that the Price Negotiator is perceived as more fluent, human-like, and a better negotiator than baseline models.
Limitations and Future Work
The authors acknowledge several areas for future work:
- Incorporating External Knowledge: The model could be improved by incorporating external knowledge about factors like delivery costs or the availability of side-offers.
- Leveraging Pre-trained Language Models: The paper suggests that using advanced pre-trained language models, such as BERT, could further enhance the understanding and generation capabilities of the agent.
- Handling More Complex Scenarios: While the current model performs well, extending it to handle more complex negotiation scenarios with multiple items or more nuanced human behaviors could be a valuable next step.
Summary updated