EAVE: Efficient Product Attribute Value Extraction via Lightweight Sparse-layer Interaction

Li Yang, Qifan Wang, Jianfeng Chi, Jiahao Liu, Jingang Wang, Fuli Feng, Zenglin Xu, Yi Fang, Lifu Huang, Dongfang Liu

Abstract

Product attribute value extraction involves identifying specific values associated with various attributes from a product profile. While existing methods prioritize model effectiveness, extraction efficiency remains a significant bottleneck in real-world scenarios where products are associated with multiple attributes. We propose EAVE (Efficient product Attribute Value Extraction), an approach utilizing lightweight sparse-layer interaction. By decoupling heavy context encoding from attribute-specific interaction, EAVE allows for context representation caching and reuse. Evaluation on two benchmarks demonstrates that EAVE achieves up to a 10x efficiency gain with marginal performance loss, particularly in long-context scenarios.

Introduction

In e-commerce, extracting structured values—such as "Leatherette" for "Material"—from product descriptions is vital for search and recommendation systems. However, a single product may have dozens of attributes. Standard Transformer-based models (e.g., BERT, T5) typically re-process the entire product description for every attribute, leading to redundant and expensive computations. EAVE addresses this by separating the heavy encoding of the product context from the lightweight interaction required for specific attributes.

Methodology

The EAVE architecture is designed to balance computational efficiency with the representational power of large-scale models. It consists of three primary components:

  1. Heavy Encoder: A large Transformer model (e.g., T5-Large) that encodes the product context and attributes independently. Since these representations do not involve cross-interaction, the context encoding can be cached and reused across all attributes for a given product.
  2. Light Encoder: A smaller Transformer (e.g., T5-Small) that jointly processes the concatenated attribute and context. This handles the necessary interaction at a significantly lower computational cost.
  3. Sparse-layer Interaction: To retain the "richness" of the heavy model within the light encoder, EAVE injects representations from the heavy encoder into the light encoder at specific intervals (sparse layers) using linear interpolation for fusion.
The EAVE architecture (a) illustrates the sparse-layer interaction between heavy and light encoders, while (b) details the representation fusion mechanism.

Experimental Evaluation

EAVE was evaluated on two major benchmarks:

  • MAVE: 3 million annotations across 2.2 million Amazon product profiles, characterized by long, structured contexts.
  • AE-110K: Over 110K examples from AliExpress, featuring shorter contexts primarily composed of product titles.

Results and Analysis

EAVE demonstrates a significant 10x reduction in computational complexity (GFLOPS) compared to state-of-the-art cross-attention models like MAVEQA. This efficiency comes with a negligible performance trade-off: F1 scores decreased by only 0.2% on MAVE and 0.9% on AE-110K.

Context Length Sensitivity

A key finding is that EAVE's effectiveness increases with context length. In scenarios with long descriptions, the ability to cache heavy context representations provides the greatest efficiency dividends. Analysis of the metrics gap shows that as context length increases, the performance difference between EAVE and fully dense heavy models diminishes, reaching parity at the longest context intervals.

Performance gap analysis across different context lengths, demonstrating that EAVE's performance converges with heavy models as context length increases.

Comparison with Efficient Baselines

EAVE outperformed other efficient text-pair models, such as MixEncoder and VIRT, by 2.34% to 5.41% in F1 score. This confirms that sparse-layer interaction is more effective than "late interaction" strategies that only merge information at the final layers.

Conclusion

EAVE provides a scalable solution for product attribute extraction by decoupling context encoding from attribute interaction. While currently optimized for extractive sequence labeling using Encoder-Decoder structures, future work will explore the application of lightweight sparse-layer interactions in generative, decoder-only architectures and other long-context tasks like document retrieval.

Summary updated

Abstract

Product attribute value extraction involves identifying the specific values associated with various attributes from a product profile. While existing methods often prioritize the development of effective models to improve extraction performance, there has been limited emphasis on extraction efficiency. However, in real-world scenarios, products are typically associated with multiple attributes, necessitating multiple extractions to obtain all corresponding values. In this work, we propose an Efficient product Attribute Value Extraction (EAVE) approach via lightweight sparse-layer interaction. Specifically, we employ a heavy encoder to separately encode the product context and attribute. The resulting non-interacting heavy representations of the context can be cached and reused for all attributes. Additionally, we introduce a light encoder to jointly encode the context and the attribute, facilitating lightweight interactions between them. To enrich the interaction within the lightweight encoder, we design a sparse-layer interaction module to fuse the non-interacting heavy representation into the lightweight encoder. Comprehensive evaluation on two benchmarks demonstrate that our method achieves significant efficiency gains with neutral or marginal loss in performance when the context is long and number of attributes is large. Our code is available \href{https://anonymous.4open.science/r/EAVE-EA18}{here}.
BibTeX
@inproceedings{yang2024eave,
  author = {Li Yang and Qifan Wang and Jianfeng Chi and Jiahao Liu and Jingang Wang and Fuli Feng and Zenglin Xu and Yi Fang and Lifu Huang and Dongfang Liu},
  title = {{EAVE: Efficient Product Attribute Value Extraction via Lightweight Sparse-layer Interaction}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2024},
  doi = {10.18653/v1/2024.findings-emnlp.80},
}
Figure