EAVE: Efficient Product Attribute Value Extraction via Lightweight Sparse-layer Interaction

Li Yang, Qifan Wang, Jianfeng Chi, Jiahao Liu, Jingang Wang, Fuli Feng, Zenglin Xu, Yi Fang, Lifu Huang, Dongfang Liu

Abstract

Product Attribute Value Extraction (AVE) is a critical task in e-commerce, identifying specific values (e.g., "Leather" for Material) from product descriptions. While state-of-the-art Transformer models are accurate, they are computationally inefficient at scale because they re-encode long product contexts for every individual attribute query. We introduce EAVE, a framework that decouples context encoding from attribute interaction. By utilizing a "Heavy-Light" architecture with sparse-layer interaction, EAVE allows for pre-computing and caching expensive context representations. Our approach achieves a 10x reduction in computational cost (GFLOPS) with a negligible impact on extraction accuracy (less than 1% F1 drop).

Introduction

In modern e-commerce platforms like Amazon or AliExpress, a single product may have dozens of attributes. Standard models jointly encode the attribute and the product description, meaning a 15-attribute product requires the system to process the same long description 15 separate times. At the scale of billions of products, this redundancy leads to massive server costs and processing delays. The challenge lies in maintaining high precision while reducing the marginal cost per attribute query.

The EAVE Framework

EAVE balances efficiency and representational power through a hybrid architecture consisting of three main components:

  1. Heavy Encoder: A large, pre-trained model (e.g., T5-Large) encodes the product context and attributes independently. Since the context encoding is attribute-agnostic in this branch, it can be pre-computed and cached.
  2. Light Encoder: A smaller, faster model (e.g., T5-Small) jointly encodes the concatenated attribute and context to capture necessary interactions.
  3. Sparse-layer Interaction: To compensate for the Light Encoder's lower capacity, EAVE injects high-quality features from the Heavy Encoder into the Light Encoder at specific layers.
The EAVE architecture uses a Heavy Encoder for cacheable context representations and a Light Encoder for efficient attribute interaction.

Fusion Mechanism

The model employs a fusion mechanism where the "non-interacting" heavy features are combined with the "interacting" light features using linear interpolation weighted by a factor $\alpha$. This allows the model to leverage the deep semantic knowledge of the heavy model without re-running its full computation for every query.

Experimental Evaluation

EAVE was evaluated on two major benchmarks:

  • MAVE: A massive Amazon dataset with 3 million annotations across 1,257 categories.
  • AE-110K: An AliExpress dataset focusing on Sports & Entertainment with over 2.7K unique attributes.

Performance and Efficiency

The primary breakthrough of EAVE is its efficiency. It achieved a 10x reduction in GFLOPS compared to standard SOTA models like AVEQA. This speedup came with minimal performance trade-offs: only a 0.2% drop in F1 score on MAVE and a 0.9% drop on AE-110K.

Sensitivity analysis of the fusion parameter alpha on the MAVE and AE-110K datasets.

As shown in the analysis of the fusion parameter $\alpha$, the model remains robust across a range of weights, though the optimal balance varies slightly by dataset. For MAVE, which features longer contexts, the efficiency gains are most pronounced as the cost of re-encoding descriptions is the primary bottleneck.

Key Insights

  • Specialized Models vs. LLMs: In comparative tests, a fine-tuned EAVE model outperformed LLaMA-2 7B in F1 score while being orders of magnitude faster. This suggests that for specific extractive tasks, specialized lightweight architectures are superior to general-purpose large language models.
  • Layer Mapping: The authors found that fusing every layer is unnecessary. For fine-tuned models, interacting at the last layer of the heavy encoder often yields the best results, simplifying the architecture further.

Conclusion

EAVE demonstrates that decoupling context and attribute encoding is a viable path toward sustainable, large-scale e-commerce AI. By caching the "heavy" work and performing only "light" interactions per query, EAVE provides a production-ready solution that maintains high accuracy while drastically reducing computational overhead.

Summary updated

Abstract

Product attribute value extraction involves identifying the specific values associated with various attributes from a product profile. While existing methods often prioritize the development of effective models to improve extraction performance, there has been limited emphasis on extraction efficiency. However, in real-world scenarios, products are typically associated with multiple attributes, necessitating multiple extractions to obtain all corresponding values. In this work, we propose an Efficient product Attribute Value Extraction (EAVE) approach via lightweight sparse-layer interaction. Specifically, we employ a heavy encoder to separately encode the product context and attribute. The resulting non-interacting heavy representations of the context can be cached and reused for all attributes. Additionally, we introduce a light encoder to jointly encode the context and the attribute, facilitating lightweight interactions between them. To enrich the interaction within the lightweight encoder, we design a sparse-layer interaction module to fuse the non-interacting heavy representation into the lightweight encoder. Comprehensive evaluation on two benchmarks demonstrate that our method achieves significant efficiency gains with neutral or marginal loss in performance when the context is long and number of attributes is large. Our code is available \href{https://anonymous.4open.science/r/EAVE-EA18}{here}.
BibTeX
@inproceedings{yang2024eave,
  author = {Li Yang and Qifan Wang and Jianfeng Chi and Jiahao Liu and Jingang Wang and Fuli Feng and Zenglin Xu and Yi Fang and Lifu Huang and Dongfang Liu},
  title = {{EAVE: Efficient Product Attribute Value Extraction via Lightweight Sparse-layer Interaction}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2024},
  doi = {10.18653/v1/2024.findings-emnlp.80},
}
Figure