Introduction
In the competitive landscape of e-commerce, detailed product information is a critical driver of customer conversion. When product descriptions are incomplete, potential buyers often abandon their carts. While automated Product Question Generation (PQG) systems aim to bridge this gap by prompting sellers for missing details, current state-of-the-art models frequently fall into a "genericity trap." These models tend to produce safe, repetitive questions like "What are the dimensions?" regardless of the product category. This lack of global diversity limits the utility of AI-driven clarification.
To address this, researchers from eBay and Technion have introduced Learning-to-Diversify (LTD), a novel fine-tuning framework designed to encourage models to explore the "long tail" of product-specific information needs without sacrificing factual relevance.
The LTD Framework: Diversifying Latent Representations
The core innovation of LTD lies in moving beyond simple output-level diversity (like beam search) to internal representation-level diversification. The framework utilizes a bi-branch architecture built upon the T5 (Text-to-Text Transfer Transformer) model.
As illustrated in Figure 1, the model is trained using a product description (context) and a pair of distinct ground-truth questions. The training process optimizes two primary objectives:
- Conditional Generation Loss (CGLoss): A standard objective to maximize the likelihood of generating the target questions.
- Diversity Regularization (DivLoss): This loss term measures the cosine similarity between the latent representations of the two questions within the Transformer's decoder.
By penalizing high similarity between these internal states, the model is forced to learn a richer, more flexible mapping between product features and potential inquiries. At inference time, the model operates as a standard T5, incurring no additional computational cost while producing more varied outputs.
Experimental Evaluation
The researchers evaluated the T5+LTD approach using four diverse categories from the Amazon dataset: Home & Kitchen, Office Products, Sports & Outdoors, and Electronics. These datasets provided a robust testing ground with varying volumes of user-submitted questions.
Key Results
The T5+LTD model was benchmarked against several baselines, including vanilla seq2seq (MLE), mixture-of-experts (hMup), and the previous state-of-the-art, KPCNet.
- Substantial Diversity Gains: The model demonstrated significant improvements in global diversity metrics (Dist-n). For instance, in the "Office Products" category, global diversity increased by up to 41% compared to standard T5.
- Preserved Relevance: A common trade-off in text generation is that increased diversity often leads to decreased accuracy. However, T5+LTD maintained or even slightly improved relevance scores, such as BLEU and BERTScore, across all categories.
- Qualitative Superiority: While baseline models often generated semantically identical questions for the same product, T5+LTD successfully produced distinct, specific inquiries (e.g., asking about both "water bottle compatibility" and "mist settings" for a humidifier).
Figure 3 highlights the impact of the LTD approach across different product domains, showing how the model successfully expands the "vocabulary" of generated questions compared to baseline methods.
Conclusion and Future Directions
The Learning-to-Diversify approach effectively mitigates the genericity bias inherent in large pre-trained Transformers. By diversifying the latent space during training, the model learns to capture multiple facets of a product's description.
Future work may involve extending this framework to decoder-only architectures like GPT and conducting large-scale human evaluations to further quantify the practical utility of these more specific questions in real-world e-commerce environments.
Summary updated