Multi-Label Zero-Shot Product Attribute-Value Extraction

Jiaying Gong, Hoda Eldardiry

TL;DR

HyperPAVE is a new framework that extracts product attributes (like "color: crimson" or "connectivity: wireless") for new items even when no labeled training data exists for those specific features. By modeling complex user behaviors and product relationships as a heterogeneous hypergraph, the system can "guess" the attributes of new products with higher accuracy and significantly lower computational cost than traditional Large Language Models (LLMs).

What Problem Does This Paper Address?

In the world of e-commerce, detailed product attributes are the backbone of search and recommendation. However, thousands of new products enter the market daily. Traditional supervised learning models are great at identifying attributes they’ve seen before, but they hit a wall when faced with "unseen" attribute-value pairs.

Manually labeling this data is a bottleneck—it's slow, expensive, and never-ending. While some researchers have tried using generative LLMs (like T5 or BART) to solve this, these models are computationally "heavy" and often struggle to extract attributes that aren't explicitly written in the product title. The authors identify a "zero-shot" gap: how do we accurately predict attributes for new products and new categories without needing a mountain of new labels?

Methods

The authors introduce HyperPAVE, which shifts the problem from text classification to inductive link prediction on a graph.

  1. Heterogeneous Hypergraph Construction: Instead of simple one-to-one connections (Product A is in Category B), they use hyperedges that can connect multiple nodes at once. This captures "higher-order" relations, such as user behavior (e.g., a single hyperedge connecting all products a specific user "also viewed") and product inventory (connecting a product to all its potential aspects).
  2. Enriched Embeddings: To give the graph "brains," they use a fine-tuned BERT encoder for product titles and descriptions. Interestingly, they use GPT-2 to expand short attribute names (like "connectivity: wireless") into detailed descriptive sentences before encoding them, ensuring the model understands the context of the attribute.
  3. Inductive Link Prediction: Most graph models are "transductive," meaning they can only reason about nodes they saw during training. HyperPAVE is "inductive"—it learns how to connect node features. This allows it to take a brand-new product node and a brand-new attribute node and calculate the probability of a link between them based on their learned representations.

Datasets

The model was evaluated using MAVE, a massive public e-commerce dataset derived from Amazon reviews. The researchers focused on ten diverse categories, including Arts, Books, Cellphones, and Grocery.

To simulate a true zero-shot environment, they carefully split the data so that the "unseen" attributes and products in the test set had zero overlap with the training set. This forced the model to rely on its ability to generalize rather than its memory.

Key Insights and Ideas

  • User Behavior is a Goldmine: One of the most striking findings was that user behavior data (the "also buy" and "also view" hyperedges) was often more influential than the actual product inventory data. Seeing what else people look at provides a latent semantic map that text alone cannot provide.
  • Hypergraphs > Simple Graphs: Standard graphs use binary edges (A connects to B). By using hypergraphs, the model can represent a user’s entire session as a single mathematical entity, preserving the "higher-order" context of how products relate in groups.
  • The Power of Description: Simply encoding the word "wireless" isn't enough. By using GPT-2 to "hallucinate" a definition for an attribute, the model gains a much richer semantic anchor, which is crucial when trying to link that attribute to a new product.

Key Results

HyperPAVE consistently outperformed classification-based, generation-based, and standard graph-based baselines across all ten categories.

  • Accuracy: In the "Books" category, HyperPAVE achieved an F1 score of 49.75%, nearly doubling the performance of BERT-based classification models (24.11%) and significantly beating generative models like BART (27.88%).
  • Efficiency: This is where the model shines for engineers. HyperPAVE is much "lighter" than LLMs. It required only 1405MB of GPU memory compared to the 8305MB required by BART.
  • Scalability: As the dataset size grows, the training time for HyperPAVE scales much more gracefully (linearly) compared to the steep exponential-like curves of fine-tuning generative transformers.

Limitations and Future Work

While HyperPAVE is a leap forward, the authors note a few areas for growth:

  • Text-Only: Currently, the model relies on text. In e-commerce, images are vital. Future versions aim to incorporate multimodal features (product images) as node attributes.
  • Static Nature: The current graph is a snapshot in time. The authors suggest moving toward dynamic graphs that include timestamps to adapt to rapidly shifting market trends and seasonal product cycles.
  • Negative Sampling: The model relies on negative sampling to learn; refining how "unlikely" attribute-product pairs are chosen could further sharpen the model's decision boundaries.

Summary updated

Abstract

E-commerce platforms must provide detailed product descriptions (attribute values) for effective product search and recommendation. However, attribute value information is typically not available for new products. To predict unseen attribute values, large quantities of labeled training data are needed to train a traditional supervised learning model. Typically, it is difficult, time-consuming, and costly to manually label large quantities of new product profiles. In this paper, we propose a novel method to efficiently and effectively extract unseen attribute values from new products in the absence of labeled data (zero-shot setting). We propose HyperPAVE, a multi-label zero-shot attribute value extraction model that leverages inductive inference in heterogeneous hypergraphs. In particular, our proposed technique constructs heterogeneous hypergraphs to capture complex higher-order relations (i.e. user behavior information) to learn more accurate feature representations for graph nodes. Furthermore, our proposed HyperPAVE model uses an inductive link prediction mechanism to infer future connections between unseen nodes. This enables HyperPAVE to identify new attribute values without the need for labeled training data. We conduct extensive experiments with ablation studies on different categories of the MAVE dataset. The results demonstrate that our proposed HyperPAVE model significantly outperforms existing classification-based, generation-based and graph-based models for attribute value extraction in the zero-shot setting.
BibTeX
@inproceedings{gong2024multi,
  author = {Jiaying Gong and Hoda Eldardiry},
  title = {{Multi-Label Zero-Shot Product Attribute-Value Extraction}},
  booktitle = {WWW},
  year = {2024},
  url = {https://openreview.net/forum?id=7hfssuKQ8P},
}
Figure