Learning to Ask: Conversational Product Search via Representation Learning

Jie Zou, Jimmy Xiangji Huang, Zhaochun Ren, Evangelos Kanoulas

TL;DR

ConvPS is a unified generative framework for conversational product search that jointly learns semantic representations for users, queries, items, and dialogue history. By combining these embeddings with sophisticated "learning to ask" strategies—including Gaussian Processes—the system effectively bridges the "vocabulary gap" and significantly outperforms both traditional search and existing conversational baselines.

What Problem Does This Paper Address?

Traditional e-commerce search is often a frustrating experience. Users might not know the exact technical terms for a product (the vocabulary mismatch problem), and static search engines struggle to capture the nuances of a user's evolving preferences.

While "conversational" search aims to fix this by asking clarifying questions (e.g., "What brand do you prefer?"), existing models have major flaws:

  1. Independence: They often model the conversation separately from the user's long-term history or the specific query.
  2. Hard-matching: They rely on exact keyword matches, failing if a user says "outstanding" when the product description uses "excellent."
  3. Inefficient Questioning: They often ask redundant or unhelpful questions that don't actually narrow down the search space quickly.

Methods

The authors introduce ConvPS (Conversational Product Search), which treats search as a joint representation learning task.

  • Unified Generative Framework: Instead of separate modules, ConvPS learns a single latent semantic space where users, queries, items, and conversation "slots" (aspects like brand, color, or size) all exist as vectors.
  • Soft-Matching: By using embeddings, the system performs "soft-matching." It calculates the probability of an item being the "target" based on how close its vector is to the combined vector of the user, their query, and the feedback received during the conversation.
  • Modeling Negative Feedback: The model explicitly learns representations for negative feedback (e.g., "I don't want a Sony"). This is a "strong signal" often ignored by simpler models.
  • Learning to Ask: The paper proposes four strategies to decide which question to ask next:
  • GBS (Generalized Binary Search): A greedy approach that tries to split the remaining item pool in half.
  • LinRel & Gaussian Processes (GP): Advanced "explore-exploit" strategies (using UCB and Expected Improvement) that balance asking questions to learn more about the user versus asking questions to confirm what the system already suspects.

Datasets

The researchers evaluated ConvPS using:

  • Amazon Product Datasets: Specifically three categories of varying sizes: Movies & TV, Health & Personal Care, and Cell Phones & Accessories.
  • Simulated Conversations: Since no large-scale "natural" conversational search dataset exists, they simulated user feedback using aspect-value pairs extracted from real product reviews.
  • Online User Study: They built a web system and recruited 26 crowd workers to interact with the model in real-time to validate the simulation results.

Key Insights and Ideas

  • The Power of Unity: The "aha moment" is that modeling the conversation inside the same embedding space as the user and the product is much more effective than treating it as an external filter.
  • Negative Feedback is Gold: Knowing what a user doesn't want is often more informative for a search engine than a vague positive preference, especially in a massive catalog.
  • Strategic Questioning: The transition from greedy questioning (GBS) to bandit-style strategies (LinRel/GP) shows that search is essentially an optimization problem. The system needs to "explore" the user's mind before it can "exploit" its knowledge to find the right item.
  • Hierarchical Awareness: Unlike prior work that just concatenated word embeddings, ConvPS respects the hierarchical structure of how users and products relate to specific attributes (slots).

Key Results

  • Superior Accuracy: ConvPS significantly outperformed state-of-the-art baselines (like HEM and AVLEM). In the "Cell Phones" category, it achieved a MAP of 0.235 compared to the best baseline's 0.125—nearly a 100% improvement.
  • Efficiency: The model's performance climbs steeply with each round of conversation. By the 10th question, the ranking of the target item improves dramatically (in one example, jumping from rank 2,251 to rank 3).
  • User Validation: In the live study, users found the questions easy to answer (71.4% agreement) and provided correct information 77% of the time, proving the system is viable for real-world deployment.

Limitations and Future Work

  • Template-Based Questions: Currently, the system uses templates (e.g., "What [brand] do you prefer?"). The authors suggest moving toward Natural Language Generation (NLG) to make the chatbot feel more human.
  • Handling Uncertainty: The model assumes users give a clear "yes/no" or specific value. Future versions need to handle "I'm not sure" or "Maybe something like..."
  • Integration: The "learning to ask" strategies are currently somewhat independent of the trained embeddings; the authors propose a tighter integration where the embeddings themselves guide the questioning strategy.
  • Cold Starts: The model relies on historical user data; more work is needed to handle entirely new users with no purchase history.

Summary updated

Abstract

Online shopping platforms, such as Amazon and AliExpress, are increasingly prevalent in society, helping customers purchase products conveniently. With recent progress in natural language processing, researchers and practitioners shift their focus from traditional product search to conversational product search. Conversational product search enables user-machine conversations and through them collects explicit user feedback that allows to actively clarify the users’ product preferences. Therefore, prospective research on an intelligent shopping assistant via conversations is indispensable. Existing publications on conversational product search either model conversations independently from users, queries, and products or lead to a vocabulary mismatch. In this work, we propose a new conversational product search model, ConvPS, to assist users in locating desirable items. The model is first trained to jointly learn the semantic representations of user, query, item, and conversation via a unified generative framework. After learning these representations, they are integrated to retrieve the target items in the latent semantic space. Meanwhile, we propose a set of greedy and explore-exploit strategies to learn to ask the user a sequence of high-performance questions for conversations. Our proposed ConvPS model can naturally integrate the representation learning of the user, query, item, and conversation into a unified generative framework, which provides a promising avenue for constructing accurate and robust conversational product search systems that are flexible and adaptive. Experimental results demonstrate that our ConvPS model significantly outperforms state-of-the-art baselines.
BibTeX
@article{zou2023learning,
  author = {Jie Zou and Jimmy Xiangji Huang and Zhaochun Ren and Evangelos Kanoulas},
  title = {{Learning to Ask: Conversational Product Search via Representation Learning}},
  journal = {ACM Trans. Information Systems},
  year = {2023},
  url = {https://openreview.net/forum?id=Dtmm7IYpqg},
}

Loading related content...

Figure