Abstract
Product Attribute Value Extraction (AVE) is a critical task in e-commerce, identifying specific values (e.g., "Leather" for Material) from product descriptions. While state-of-the-art Transformer models are accurate, they are computationally inefficient at scale because they re-encode long product contexts for every individual attribute query. We introduce EAVE, a framework that decouples context encoding from attribute interaction. By utilizing a "Heavy-Light" architecture with sparse-layer interaction, EAVE allows for pre-computing and caching expensive context representations. Our approach achieves a 10x reduction in computational cost (GFLOPS) with a negligible impact on extraction accuracy (less than 1% F1 drop).
Introduction
In modern e-commerce platforms like Amazon or AliExpress, a single product may have dozens of attributes. Standard models jointly encode the attribute and the product description, meaning a 15-attribute product requires the system to process the same long description 15 separate times. At the scale of billions of products, this redundancy leads to massive server costs and processing delays. The challenge lies in maintaining high precision while reducing the marginal cost per attribute query.
The EAVE Framework
EAVE balances efficiency and representational power through a hybrid architecture consisting of three main components:
- Heavy Encoder: A large, pre-trained model (e.g., T5-Large) encodes the product context and attributes independently. Since the context encoding is attribute-agnostic in this branch, it can be pre-computed and cached.
- Light Encoder: A smaller, faster model (e.g., T5-Small) jointly encodes the concatenated attribute and context to capture necessary interactions.
- Sparse-layer Interaction: To compensate for the Light Encoder's lower capacity, EAVE injects high-quality features from the Heavy Encoder into the Light Encoder at specific layers.
Fusion Mechanism
The model employs a fusion mechanism where the "non-interacting" heavy features are combined with the "interacting" light features using linear interpolation weighted by a factor $\alpha$. This allows the model to leverage the deep semantic knowledge of the heavy model without re-running its full computation for every query.
Experimental Evaluation
EAVE was evaluated on two major benchmarks:
- MAVE: A massive Amazon dataset with 3 million annotations across 1,257 categories.
- AE-110K: An AliExpress dataset focusing on Sports & Entertainment with over 2.7K unique attributes.
Performance and Efficiency
The primary breakthrough of EAVE is its efficiency. It achieved a 10x reduction in GFLOPS compared to standard SOTA models like AVEQA. This speedup came with minimal performance trade-offs: only a 0.2% drop in F1 score on MAVE and a 0.9% drop on AE-110K.
As shown in the analysis of the fusion parameter $\alpha$, the model remains robust across a range of weights, though the optimal balance varies slightly by dataset. For MAVE, which features longer contexts, the efficiency gains are most pronounced as the cost of re-encoding descriptions is the primary bottleneck.
Key Insights
- Specialized Models vs. LLMs: In comparative tests, a fine-tuned EAVE model outperformed LLaMA-2 7B in F1 score while being orders of magnitude faster. This suggests that for specific extractive tasks, specialized lightweight architectures are superior to general-purpose large language models.
- Layer Mapping: The authors found that fusing every layer is unnecessary. For fine-tuned models, interacting at the last layer of the heavy encoder often yields the best results, simplifying the architecture further.
Conclusion
EAVE demonstrates that decoupling context and attribute encoding is a viable path toward sustainable, large-scale e-commerce AI. By caching the "heavy" work and performing only "light" interactions per query, EAVE provides a production-ready solution that maintains high accuracy while drastically reducing computational overhead.
Summary updated