JPAVE: A Generation and Classification-based Model for Joint Product Attribute Prediction and Value Extraction

Zhongfen Deng, Hao Peng, Tao Zhang, Shuaiqi Liu, Wenting Zhao, Yibo Wang, Philip S. Yu

TL;DR

JPAVE is a multi-task learning framework that moves away from traditional, position-dependent sequence labeling for product attribute extraction. By combining attribute prediction with either value generation (using a copy mechanism) or classification, it achieves state-of-the-art performance while remaining robust to "data shifting"—where the same product information is presented in different styles or structures across e-commerce platforms.

What Problem Does This Paper Address?

In e-commerce, extracting attribute-value pairs (e.g., Attribute: "Color", Value: "Midnight Blue") from product titles and descriptions is vital for search and recommendation. Most existing models treat this as a sequence labeling task (like NER) or a Question Answering task.

However, these approaches have three major flaws:

  1. Dependency on Position: They require expensive, manually annotated data that specifies exactly where a value starts and ends in a string.
  2. Data Discrepancy (Data Shifting): Models trained on one site's format often fail on another because the sentence structure or word order changes, even if the core information remains the same.
  3. Poor Zero-Shot Ability: They struggle to identify new, "unseen" values that weren't present in the training set.

Methods

The authors propose JPAVE (Joint Product Attribute Prediction and Value Extraction), which consists of a shared text encoder (Bi-GRU) and an attribute predictor, paired with one of two value-extraction variants:

  • JPAVE-GEN (Open-World): Uses a GRU-based decoder with a copy mechanism (Pointer-Generator). This allows the model to either generate a word from its vocabulary or "copy" a word directly from the input text. This is crucial for identifying brand-new values (like a new color name) that appear in the description.
  • JPAVE-CLS (Closed-World): Uses a value attention module to match input text against a pre-defined set of values. This is ideal for scenarios where the possible values are known and fixed.

Key Innovation: Because the model doesn't rely on BIO-tagging (Beginning, Inside, Outside) or specific index positions, it is position-agnostic. It focuses on the semantic presence of information rather than its coordinates in a string.

Datasets

The model was evaluated on MEPAVE, a large-scale Chinese e-commerce dataset.

  • Statistics: 26 attributes, 2,129 unique values, and over 71,000 training instances.
  • While MEPAVE is technically a multimodal dataset (including images), the authors focused solely on the textual descriptions to demonstrate the strength of their NLP architecture.

Key Insights and Ideas

  • Structure is Noise: The authors argue that for attribute extraction, sentence structure is often "noise" that causes models to overfit. By using a copy mechanism and attention, JPAVE ignores the "fluff" and homes in on the relevant tokens.
  • Robustness through Permutation: To prove the model is position-agnostic, the researchers tested it on a "permuted" dataset where word order was randomly shuffled. While traditional sequence labelers would break, JPAVE’s performance barely dipped, proving it learns the content rather than the template.
  • Efficiency Matters: JPAVE is remarkably lightweight. With only 12.3M parameters, it outperformed much larger models like T5-base (220M) and GPT-2 (124M) on this specific task, making it more viable for high-throughput production environments.

Key Results

JPAVE outperformed all baselines, including multimodal models that had access to product images.

  • Overall Performance: JPAVE-GEN achieved an F1 score of 98.40% for attribute prediction and 97.51% for value extraction.
  • Zero-Shot Success: In the open-world scenario, JPAVE-GEN achieved an 83.15% F1 score on unseen values, with a perfect 100% precision (meaning when it identified a new value, it was always correct).
  • Comparison: It beat the previous state-of-the-art (M-JAVE) by roughly 8% in attribute prediction and 10% in value extraction.

Limitations and Future Work

  • Language Scope: The current study focuses on a Chinese dataset. The authors acknowledge that exploring cross-lingual attribute extraction is a necessary next step, as different languages have different syntactic characteristics.
  • Value Source: The copy mechanism assumes the value is explicitly written in the text. It might struggle with "implicit" attributes where the value must be inferred (e.g., inferring the attribute "Season: Summer" from the word "breathable").
  • Multimodality: While JPAVE beat multimodal baselines using only text, the authors suggest that future versions could integrate visual features to further boost performance in cases where the text is sparse.

Summary updated

Abstract

Product attribute value extraction is an important task in e-Commerce which can help several downstream applications such as product search and recommendation. Most previous models handle this task using sequence labeling or question answering method which rely on the sequential position information of values in the product text and are vulnerable to data discrepancy between training and testing. This limits their generalization ability to real-world scenario in which each product can have multiple descriptions across various shopping platforms with different composition of text and style. They also have limited zero-shot ability to new values. In this paper, we propose a multi-task learning model with value generation/classification and attribute prediction called JPAVE to predict values without the necessity of position information of values in the text. Furthermore, the copy mechanism in value generator and the value attention module in value classifier help our model address the data discrepancy issue by only focusing on the relevant part of input text and ignoring other information which causes the discrepancy issue such as sentence structure in the text. Besides, two variants of our model are designed for open-world and closed-world scenarios. In addition, copy mechanism introduced in the first variant based on value generation can improve its zero-shot ability for identifying unseen values. Experimental results on a public dataset demonstrate the superiority of our model compared with strong baselines and its generalization ability of predicting new values.
BibTeX
@inproceedings{deng2023jpave,
  author = {Zhongfen Deng and Hao Peng and Tao Zhang and Shuaiqi Liu and Wenting Zhao and Yibo Wang and Philip S. Yu},
  title = {{JPAVE: A Generation and Classification-based Model for Joint Product Attribute Prediction and Value Extraction}},
  booktitle = {IEEE Big Data},
  year = {2023},
  doi = {10.1109/bigdata59044.2023.10386204},
}

Loading related content...

Figure