Product Description and QA Assisted Self-Supervised Opinion Summarization

Tejpalsingh Siledar, Rupasai Rangaraju, Sankara Sri Raghava Ravindra Muddu, Suman Banerjee, Amey Patil, Sudhanshu Singh, Muthusamy Chelliah, Nikesh Garera, Swaprava Nath, Pushpak BhattacharyyaNAACL 2024 · DOI

Abstract

Traditional opinion summarization focuses primarily on customer reviews, often neglecting critical technical specifications and compatibility information. We present MEDOS, a Multi-Encoder Decoder framework for Opinion Summarization that integrates product descriptions and Q&A data. Using a novel self-supervised training strategy called Synthetic Dataset Creation (SDC), MEDOS learns to fuse subjective reviews with objective product facts. Our approach achieves a 14.5% improvement in ROUGE scores over state-of-the-art methods, producing summaries that are more informative, technical, and contextually accurate.

Introduction

In e-commerce, opinion summarization aims to distill vast quantities of user reviews into concise, readable summaries. However, reviews are frequently subjective and lack technical precision. For instance, a reviewer might mention a software incompatibility without specifying the operating system, or praise a product's ergonomics without identifying the specific model version.

This research addresses two primary challenges in the field:

  1. Information Silos: Technical specifications in product descriptions and specific compatibility details in Q&A sections are typically ignored by existing summarization models.
  2. Data Scarcity: There is a lack of high-quality, supervised training data that pairs reviews, descriptions, and Q&A with human-written summaries.

Methodology

The proposed framework introduces two core innovations to bridge these gaps: Synthetic Dataset Creation and the MEDOS architecture.

Synthetic Dataset Creation (SDC)

To overcome the lack of labeled data, we employ a self-supervised pipeline. We identify a "pseudo-summary" by selecting a review from the pool that is semantically closest to both the product description and the Q&A data. This "bridge" review serves as the target for training. The model is then tasked with reconstructing this target using the remaining reviews, the description, and the Q&A as inputs. This process forces the model to learn how to extract factual details from specifications and context from Q&A to align with the sentiment found in reviews.

MEDOS Architecture

Unlike standard models that concatenate all inputs into a single string, MEDOS utilizes a multi-encoder approach to maintain the distinct characteristics of each data source.

The MEDOS architecture features separate encoders for reviews, descriptions, and Q&A, feeding into a single decoder.

The framework employs dedicated encoders for reviews, descriptions, and Q&A. A gated fusion mechanism dynamically weights the contribution of the description and Q&A during the decoding process. This allows the model to selectively incorporate brand names or specific technical fixes into the final summary.

Experiments and Results

We evaluated MEDOS across three major datasets: Amazon, Oposum+, and Flipkart. To provide a more rigorous benchmark, we extended these datasets using GPT-3.5 to create "gold" summaries that specifically incorporate description and Q&A data (GPT-RDQ).

Quantitative Performance

MEDOS consistently outperformed baseline models across all metrics. On the Amazon GPT-RDQ dataset, MEDOS achieved significant gains in ROUGE-1, ROUGE-2, and ROUGE-L scores compared to previous state-of-the-art models like MultimodalSum and BART-concat.

Performance comparison of MEDOS against various baselines on Amazon datasets.

Key Insights

  • Complementary Information: Product descriptions provide the "nouns" (technical specs, model numbers), while reviews provide the "adjectives" (sentiment, user experience). MEDOS effectively slots these technical details into the opinionated flow of the summary.
  • Efficiency vs. Scale: A specialized 0.8B parameter MEDOS model performed comparably to much larger general-purpose models like Claude-2 (130B) and Llama-2 (70B), demonstrating the value of architectural specialization over raw scale.
  • Human Evaluation: In human-led Best-Worst Scaling tests, MEDOS was rated significantly higher in coherence (0.41) and fluency (0.50) than competing models.

Conclusion and Future Work

MEDOS demonstrates that incorporating diverse information sources—beyond just reviews—significantly enhances the quality of product summaries. While the model shows a marked improvement in factual accuracy and technical depth, challenges remain regarding long-context handling and occasional hallucinations. Future work will explore expanding the framework to include multimodal inputs, such as product images, to provide an even more comprehensive "360-degree" product overview.

Summary updated

BibTeX
@inproceedings{siledar2024product,
  author = {Tejpalsingh Siledar and Rupasai Rangaraju and Sankara Sri Raghava Ravindra Muddu and Suman Banerjee and Amey Patil and Sudhanshu Singh and Muthusamy Chelliah and Nikesh Garera and Swaprava Nath and Pushpak Bhattacharyya},
  title = {{Product Description and QA Assisted Self-Supervised Opinion Summarization}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2024},
  doi = {10.18653/v1/2024.findings-naacl.150},
}

Loading related content...

Figure