TL;DR
ConvPS is a unified generative framework for conversational product search that jointly learns semantic representations for users, queries, items, and dialogue history. By combining these embeddings with sophisticated "learning to ask" strategies—including Gaussian Processes—the system effectively bridges the "vocabulary gap" and significantly outperforms both traditional search and existing conversational baselines.
What Problem Does This Paper Address?
Traditional e-commerce search is often a frustrating experience. Users might not know the exact technical terms for a product (the vocabulary mismatch problem), and static search engines struggle to capture the nuances of a user's evolving preferences.
While "conversational" search aims to fix this by asking clarifying questions (e.g., "What brand do you prefer?"), existing models have major flaws:
- Independence: They often model the conversation separately from the user's long-term history or the specific query.
- Hard-matching: They rely on exact keyword matches, failing if a user says "outstanding" when the product description uses "excellent."
- Inefficient Questioning: They often ask redundant or unhelpful questions that don't actually narrow down the search space quickly.
Methods
The authors introduce ConvPS (Conversational Product Search), which treats search as a joint representation learning task.
- Unified Generative Framework: Instead of separate modules, ConvPS learns a single latent semantic space where users, queries, items, and conversation "slots" (aspects like brand, color, or size) all exist as vectors.
- Soft-Matching: By using embeddings, the system performs "soft-matching." It calculates the probability of an item being the "target" based on how close its vector is to the combined vector of the user, their query, and the feedback received during the conversation.
- Modeling Negative Feedback: The model explicitly learns representations for negative feedback (e.g., "I don't want a Sony"). This is a "strong signal" often ignored by simpler models.
- Learning to Ask: The paper proposes four strategies to decide which question to ask next:
- GBS (Generalized Binary Search): A greedy approach that tries to split the remaining item pool in half.
- LinRel & Gaussian Processes (GP): Advanced "explore-exploit" strategies (using UCB and Expected Improvement) that balance asking questions to learn more about the user versus asking questions to confirm what the system already suspects.
Datasets
The researchers evaluated ConvPS using:
- Amazon Product Datasets: Specifically three categories of varying sizes: Movies & TV, Health & Personal Care, and Cell Phones & Accessories.
- Simulated Conversations: Since no large-scale "natural" conversational search dataset exists, they simulated user feedback using aspect-value pairs extracted from real product reviews.
- Online User Study: They built a web system and recruited 26 crowd workers to interact with the model in real-time to validate the simulation results.
Key Insights and Ideas
- The Power of Unity: The "aha moment" is that modeling the conversation inside the same embedding space as the user and the product is much more effective than treating it as an external filter.
- Negative Feedback is Gold: Knowing what a user doesn't want is often more informative for a search engine than a vague positive preference, especially in a massive catalog.
- Strategic Questioning: The transition from greedy questioning (GBS) to bandit-style strategies (LinRel/GP) shows that search is essentially an optimization problem. The system needs to "explore" the user's mind before it can "exploit" its knowledge to find the right item.
- Hierarchical Awareness: Unlike prior work that just concatenated word embeddings, ConvPS respects the hierarchical structure of how users and products relate to specific attributes (slots).
Key Results
- Superior Accuracy: ConvPS significantly outperformed state-of-the-art baselines (like HEM and AVLEM). In the "Cell Phones" category, it achieved a MAP of 0.235 compared to the best baseline's 0.125—nearly a 100% improvement.
- Efficiency: The model's performance climbs steeply with each round of conversation. By the 10th question, the ranking of the target item improves dramatically (in one example, jumping from rank 2,251 to rank 3).
- User Validation: In the live study, users found the questions easy to answer (71.4% agreement) and provided correct information 77% of the time, proving the system is viable for real-world deployment.
Limitations and Future Work
- Template-Based Questions: Currently, the system uses templates (e.g., "What [brand] do you prefer?"). The authors suggest moving toward Natural Language Generation (NLG) to make the chatbot feel more human.
- Handling Uncertainty: The model assumes users give a clear "yes/no" or specific value. Future versions need to handle "I'm not sure" or "Maybe something like..."
- Integration: The "learning to ask" strategies are currently somewhat independent of the trained embeddings; the authors propose a tighter integration where the embeddings themselves guide the questioning strategy.
- Cold Starts: The model relies on historical user data; more work is needed to handle entirely new users with no purchase history.
Summary updated