MAVE: A Product Dataset for Multi-source Attribute Value Extraction

Li Yang, Qifan Wang, Zac Yu, Anand Kulkarni, Sumit Sanghai, Bin Shu, Jon Elsas, Bhargav Kanagal

TL;DR

Researchers at Google have introduced MAVE, the largest and most diverse public dataset for product attribute value extraction, featuring 2.2 million products and 3 million annotations. Alongside the dataset, they propose MAVEQA, a model that leverages global-local attention to effectively extract information from long, multi-source product descriptions where standard models typically struggle.

What Problem Does This Paper Address?

In e-commerce, structured data is king. Knowing that a laptop has "7 Hours" of "Battery Life" or an "11.6-Inch" "Screen Size" is essential for search filters, recommendation engines, and comparison tools. However, this information is often buried in messy, unstructured text across various parts of a product page—titles, bulleted lists, and long descriptions.

Existing research in Attribute Value Extraction (AVE) has been hampered by three main issues:

  1. Small/Private Datasets: Most datasets are either tiny (a few thousand examples) or kept behind corporate walls.
  2. Single-Source Focus: Many models only look at product titles, ignoring the rich details found in descriptions or specifications.
  3. The "Long Sequence" Problem: Standard Transformer models (like BERT) are limited to 512 tokens. Detailed product pages often exceed this limit, leading to data truncation and lost information.

Methods

The authors tackle these challenges with a new model architecture called MAVEQA.

Instead of treating extraction as a simple tagging task, they frame it as a Question Answering (QA) problem: given an attribute (the "question") and the product text (the "context"), find the "answer" span.

To handle long, multi-source text, MAVEQA uses the Extended Transformer Construction (ETC) backbone. As shown in Figure 4, the model uses a "global-local" attention mechanism:

  • Global Tokens: Represent high-level summaries of different sources (e.g., one token for the title, one for the description).
  • Long Tokens: Represent the actual words in the text.
  • Efficiency: Instead of every word attending to every other word (which is computationally expensive), words attend to a local neighborhood and to the global tokens. This allows the model to process sequences of 1024 tokens or more efficiently.
Figure 4: The MAVEQA architecture uses global tokens to summarize different information sources and local attention to handle long sequences.

Datasets

The star of the paper is the MAVE dataset. It was curated from millions of Amazon product pages using a rigorous pipeline involving an ensemble of models and human-verified rules to ensure a high precision bar (97.8%).

  • Scale: 2.2 million products across 1,257 categories.
  • Diversity: 2,535 unique category-attribute pairs.
  • Multi-source: Unlike previous datasets, MAVE includes titles, descriptions, features, and more.
  • Benchmarks: The authors provide specific subsets for evaluating "Head" attributes (common), "Tail" attributes (rare), and "Zero-shot" attributes (unseen during training).
Figure 3: Distribution of categories and attributes in MAVE. The inner circle shows categories like "Shoes" or "Books," while the outer circle shows specific attributes like "Heel Height" or "Fiction Form."

Key Insights and Ideas

  • The Multi-Source Advantage: Information is rarely in just one place. As Figure 1 illustrates, "Battery Life" might only appear in the specifications, while "Brand" is in the title. MAVE is designed to force models to look everywhere.
  • Tail Attributes are "Easy" but Rare: Interestingly, the model achieved nearly 100% F1 scores on some "tail" attributes (like "Black Tea Variety"). Even though there were few training examples, the linguistic patterns for these attributes were very consistent and simple to learn.
  • Zero-Shot is the Frontier: The paper highlights how difficult it is to extract attributes the model has never seen before. While the model can generalize "Resolution" from one category to another, it completely fails on attributes like "Compatibility" if no prior examples exist, suggesting a need for better transfer learning.

Key Results

MAVEQA consistently outperformed standard baselines, particularly as the length of the product information increased.

  • Handling Length: As shown in Figure 5, when product descriptions grew beyond 512 tokens, the performance of standard models (AVEQA) dropped significantly because they had to truncate the text. MAVEQA maintained high precision and recall by processing the full context.
  • Few-Shot Learning: The model is remarkably efficient. Figure 6 shows that for many attributes, the model reaches over 90% F1 score with as few as 10 to 50 training examples.
Figure 5: Error rates (1 - Metric) across different sequence lengths. MAVEQA (orange) shows significantly lower errors than the baseline (blue) for long sequences.

Limitations and Future Work

The authors acknowledge that zero-shot extraction remains a major hurdle. For truly "difficult" attributes that require deep domain knowledge (like "Compatibility"), current models still struggle to generalize without at least a few examples.

Additionally, they note that while MAVE is high-quality, it isn't perfect; some "negative" examples (where the model says a value doesn't exist) might actually be missing annotations. Future work involves using MAVE as a pre-training task to help models learn better general-purpose product representations for search and recommendation.

Summary updated

Abstract

Attribute value extraction refers to the task of identifying values of an attribute of interest from product information. Product attribute values are essential in many e-commerce scenarios, such as customer service robots, product ranking, retrieval and recommendations. While in the real world, the attribute values of a product are usually incomplete and vary over time, which greatly hinders the practical applications. In this paper, we introduce MAVE, a new dataset to better facilitate research on product attribute value extraction. MAVE is composed of a curated set of 2.2 million products from Amazon pages, with 3 million attribute-value annotations across 1257 unique categories. MAVE has four main and unique advantages: First, MAVE is the largest product attribute value extraction dataset by the number of attribute-value examples. Second, MAVE includes multi-source representations from the product, which captures the full product information with high attribute coverage. Third, MAVE represents a more diverse set of attributes and values relative to what previous datasets cover. Lastly, MAVE provides a very challenging zero-shot test set, as we empirically illustrate in the experiments. We further propose a novel approach that effectively extracts the attribute value from the multi-source product information. We conduct extensive experiments with several baselines and show that MAVE is an effective dataset for attribute value extraction task. It is also a very challenging task on zero-shot attribute extraction. Data is available at \urlhttps://github.com/google-research-datasets/MAVE .
BibTeX
@inproceedings{yang2022mave,
  author = {Li Yang and Qifan Wang and Zac Yu and Anand Kulkarni and Sumit Sanghai and Bin Shu and Jon Elsas and Bhargav Kanagal},
  title = {{MAVE: A Product Dataset for Multi-source Attribute Value Extraction}},
  booktitle = {WSDM},
  year = {2022},
  doi = {10.1145/3488560.3498377},
}

Loading related content...

Figure