Probing Product Description Generation via Posterior Distillation

Haolan Zhan, Hainan Zhang, Hongshen Chen, Lei Shen, Zhuoye Ding, Yongjun Bao, Weipeng Yan, Yanyan Lan

Abstract

In product description generation (PDG), capturing user-cared aspects is critical for recommendation systems to improve user experience and click-through rates. High-quality customer reviews are ideal sources for mining these aspects. However, many new products (long-tailed commodities) lack sufficient reviews, posing a challenge for generation. Existing methods often rely solely on product attributes or titles, resulting in tedious content. We propose the Adaptive Posterior Distillation Transformer (APDT), which utilizes user-cared information from reviews through a teacher-student distillation framework. By forcing a student network (using only titles and attributes) to mimic a teacher network (using reviews), we enable the generation of engaging descriptions even for products without reviews. Experimental results on a large-scale Chinese dataset demonstrate our model's superiority in both automatic metrics and human evaluation.

Introduction

E-commerce platforms rely heavily on engaging product descriptions to drive consumer interest. A high-quality description goes beyond technical specifications, highlighting "user-cared" aspects—subjective qualities like comfort, ease of use, or style that resonate with potential buyers. Customer reviews are a primary source for these insights, providing a bridge between raw product data and consumer needs.

Example of product metadata and customer reviews used to generate a user-centric description.

However, a significant challenge arises with "long-tailed commodities"—new or niche products that have few or no reviews. Traditional generation models for these items often produce dry, repetitive text. To address this, we propose a method to "transfer" the wisdom found in existing reviews of popular products to the generation process for products that do not have any reviews yet.

Methodology: The Adaptive Posterior Distillation Transformer

The core innovation of APDT is a training strategy involving two roles: a Teacher (Posterior) and a Student (Prior).

Architecture Overview

The model consists of a multi-source encoder, an adaptive posterior distillation layer, and a Transformer-based decoder with a copy mechanism.

The APDT architecture, highlighting the distillation process between the Teacher (posterior) and Student (prior) networks.
  1. Multi-Source Encoder: The model uses separate self-attentive Transformer encoder branches to process product titles and attributes independently, capturing the unique semantic information of each source.
  2. Adaptive Posterior Distillation: During training, the Teacher module has access to the product title, attributes, and actual customer reviews. It learns to create a rich representation of what users value. The Student module only sees the title and attributes. Through a Kullback-Leibler (KL) divergence loss, the Student is forced to mimic the Teacher's internal representations. This allows the Student to "imagine" likely user-cared aspects during inference when reviews are unavailable.
  3. Coherence-Enhanced Loss (CoE): To prevent the model from generating generic praise or "hallucinating" irrelevant info, a custom training objective rewards the generation of words that are both frequent in the ground truth and semantically coherent with the specific product.
  4. Copy Mechanism: A standard pointer-generator component allows the model to precisely copy specific attributes (like brand names or model numbers) directly from the input into the description, ensuring factual accuracy.

Experimental Results

We introduced the JDPDG dataset, a large-scale benchmark collected from JD.com containing 345,799 pairs of item content and descriptions across categories like Shoes & Clothes, Digital, and Homing.

Performance Comparison

APDT consistently outperformed state-of-the-art baselines, including KOBE and standard Transformers. In the Shoes & Clothes category, APDT achieved a ROUGE-L of 20.41 and BLEU of 10.36, compared to KOBE's 19.07 and 9.27, respectively.

Human evaluations further validated these results. In blind tests, human graders rated APDT higher than all baselines in both Correctness (2.91/4.0) and Coherence (3.02/4.0). Qualitative case studies showed that while baseline models might only mention a laptop's color, APDT successfully added phrases like "very convenient for office" and "runs very smoothly," mimicking the helpful tone of a real customer review.

Conclusion

The Adaptive Posterior Distillation Transformer (APDT) effectively addresses the cold-start problem in product description generation. By treating reviews as a latent signal to be distilled during training, the model learns to generate engaging, user-centric descriptions for new products using only basic metadata at inference time. This approach significantly improves the quality and relevance of generated content in e-commerce recommendation systems.

Summary updated

Abstract

In product description generation (PDG), the user-cared aspect is critical for the recommendation system, which can not only improve user's experiences but also obtain more clicks. High-quality customer reviews can be considered as an ideal source to mine user-cared aspects. However, in reality, a large number of new products (known as long-tailed commodities) cannot gather sufficient amount of customer reviews, which brings a big challenge in the product description generation task. Existing works tend to generate the product description solely based on item information, i.e., product attributes or title words, which leads to tedious contents and cannot attract customers effectively. To tackle this problem, we propose an adaptive posterior network based on Transformer architecture that can utilize user-cared information from customer reviews. Specifically, we first extend the self-attentive Transformer encoder to encode product titles and attributes. Then, we apply an adaptive posterior distillation module to utilize useful review information, which integrates user-cared aspects to the generation process. Finally, we apply a Transformer-based decoding phase with copy mechanism to automatically generate the product description. Besides, we also collect a large-scare Chinese product description dataset to support our work and further research in this field. Experimental results show that our model is superior to traditional generative models in both automatic indicators and human evaluation.
BibTeX
@inproceedings{zhan2021probing,
  author = {Haolan Zhan and Hainan Zhang and Hongshen Chen and Lei Shen and Zhuoye Ding and Yongjun Bao and Weipeng Yan and Yanyan Lan},
  title = {{Probing Product Description Generation via Posterior Distillation}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  url = {https://openreview.net/forum?id=bAky9v-NWPr},
}
Figure