Abstract
Extracting accurate attribute qualities from product titles is a vital component in delivering eCommerce customers with a rewarding online shopping experience via an enriched faceted search. We demonstrate the potential of Deep Recurrent Networks in this domain, primarily models such as Bidirectional LSTMs and Bidirectional LSTM-CRF with or without an attention mechanism. These have improved overall F1 scores, as compared to the previous benchmarks by at least 0.0391, showcasing an overall precision of 97.94%, recall of 94.12%, and an F1 score of 0.9599. This achievement in significant coverage of important product facets not only shows the efficacy of deep recurrent models over previous machine learning benchmarks but also greatly enhances the overall customer experience while shopping online.
Introduction
In the competitive landscape of eCommerce, "faceted search"—the ability to filter products by specific attributes like brand, color, or material—is essential for product discovery. However, the effectiveness of these filters depends entirely on the accuracy of the underlying metadata. Traditionally, identifying these attributes from raw, often messy product titles was treated as a sequence labeling problem solved with models like Linear Chain Conditional Random Fields (CRFs) or Structured Perceptrons.
While functional, these traditional methods rely heavily on manual feature engineering, which is difficult to scale across millions of diverse products and rapidly evolving categories. Researchers at Walmart Labs have addressed this by implementing deep recurrent neural networks that learn features automatically, providing superior extraction accuracy and scalability.
Methodology
Sequence Labeling and BIO Encoding
The task of attribute extraction is framed as a Sequence-to-Sequence Labeling problem. To distinguish between different attributes and their boundaries, the authors utilized the BIO (Beginning, Inside, Outside) encoding scheme. In this system, the first token of an attribute is marked with a 'B-' prefix, subsequent tokens within the same attribute are marked with 'I-', and tokens that do not belong to any attribute of interest are marked as 'O'.
Model Architecture
The core architecture explored is the Bidirectional LSTM-CRF. This model combines three powerful components:
- Bidirectional LSTM: This layer processes the product title in both forward and backward directions. This allows the model to capture the context of a word based on both its preceding and succeeding tokens, which is critical for understanding product nomenclature.
- Embeddings: The model utilizes a hybrid approach, combining word embeddings for semantic meaning and character embeddings to capture sub-word patterns. This helps the model handle out-of-vocabulary terms, such as new or misspelled brand names.
- CRF Layer: A Conditional Random Field layer sits atop the LSTM to predict the final labels. Unlike a simple softmax, the CRF considers the dependencies between neighboring labels (e.g., ensuring an 'I-' tag follows a 'B-' tag), ensuring the output sequence is mathematically consistent.
The researchers also experimented with an Attention Mechanism to help the model focus on specific locational interests within the title, though its impact varied depending on the overall architecture.
Experimental Setup and Results
The study utilized a dataset of 61,374 product titles sourced from online catalogs, with a primary focus on the 'Brand' attribute. Labels were acquired through high-quality crowdsourcing to establish a reliable ground truth. The data was split into a 60/20/20 ratio for training, validation, and testing.
The Bi-LSTM-CRF model established a new state-of-the-art performance for the brand extraction task, significantly outperforming previous benchmarks like the Structured Perceptron:
- F1-Score: 0.9599 (an improvement of ~0.039)
- Precision: 97.94%
- Recall: 94.12%
- Label Accuracy: 99.44%
Interestingly, the study found that while the Attention Mechanism improved a standard Bi-LSTM, it did not provide additional gains when paired with a CRF layer. This suggests that in the context of short product titles, the CRF and Attention mechanisms may perform redundant functions.
Real-World Impact and Conclusion
Beyond laboratory metrics, the implementation of these deep learning models had a measurable impact on business performance. By providing more accurate attribute values, the system drove approximately 250,000 additional impressions per day during a 27-day test period. This demonstrates that moving from manual feature engineering to deep sequence labeling not only improves technical accuracy but directly enhances the customer's ability to find products, leading to higher engagement in the eCommerce ecosystem.
Summary updated