Label-Guided Learning for Item Categorization in e-Commerce

Lei Chen, Hirokazu MiyakeNAACL 2021 · DOI

TL;DR

Researchers at Rakuten have demonstrated that treating e-commerce category labels as semantic data—rather than just arbitrary IDs—significantly boosts item categorization accuracy. By using hyperbolic (Poincaré) embeddings to represent hierarchical taxonomies and employing label-specific attention mechanisms, they achieved substantial performance gains, particularly on niche product categories where training data is scarce.

What Problem Does This Paper Address?

In e-commerce, Item Categorization (IC) is the critical task of assigning a product to the correct leaf node within a vast taxonomy tree. For instance, a "tin of Matcha" must be correctly routed through levels like Beverage -> Tea -> Japanese Tea.

Example of item categorization within a hierarchical taxonomy.

This task presents three significant challenges:

  1. Short, Messy Input: Product titles are often composed of keyword strings rather than grammatical sentences, making traditional NLP parsing difficult.
  2. Massive Scale: Industrial taxonomies contain thousands of classes. Many are semantically similar (e.g., "Japanese tea" vs. "Chinese tea"), leading to high confusion rates.
  3. Underutilized Information: Traditional models often treat labels as one-hot encoded vectors or simple indices. This ignores the rich semantic information inherent in the label names themselves. The word "Tea" in a label is a powerful clue that should be leveraged to understand the product title.

Methods: Label-Guided Learning

The authors explored Label-Guided Learning, a paradigm where the model learns a joint embedding space for both input text and category labels. They evaluated two primary architectures:

  • LEAM (Label Embedding Attentive Model): A streamlined model that uses label embeddings to calculate attention weights directly on input words, highlighting relevant terms.
  • LSAN (Label-Specific Attention Network): A more sophisticated architecture employing a Bi-LSTM to capture context. It adaptively combines "self-attention" (internal text context) with "label-attention" (relevance to specific category names).

The "Aha!" Twist: Hyperbolic Embeddings

A key innovation in this research is the use of Poincaré embeddings to represent the taxonomy. Standard embeddings reside in Euclidean space, which struggles to represent hierarchical structures. As a tree grows, the number of nodes increases exponentially with depth, but the volume of Euclidean space only grows polynomially. This "cramps" the leaf nodes, making them hard to distinguish.

Comparison of tree structures in Euclidean space (a) versus the Poincaré disk (b).

In contrast, hyperbolic geometry (represented by the Poincaré disk) sees distance grow exponentially with radius. This provides the necessary "room" to separate similar leaf nodes while preserving their hierarchical relationships to parent nodes.

Datasets and Insights

The study utilized a massive real-world dataset from Rakuten, encompassing over 1 million products across four root-level genres: Catalog Gifts & Tickets, Beverages, Appliances, and Men’s Fashion.

Key insights from the experimentation include:

  • Labels as Features: Embedding the actual words in labels allows the model to "bridge" the gap for rare products. Even if a specific title-category pair is infrequent in training, the shared semantic space helps the model make accurate predictions.
  • Genre-Specific Focus: The researchers found that pre-training word embeddings on specific categories (e.g., just "Appliances") outperformed training on the entire dataset. Specialized vocabularies reduce noise and help the model learn more precise semantic clusters.

Key Results

Performance was measured using the Macro F-score to ensure fair evaluation across both popular and niche categories.

  • LSAN Superiority: The Label-Specific Attention Network consistently outperformed the simpler LEAM model across all tested genres.
  • The Poincaré Boost: Integrating hyperbolic embeddings (LSAN-Poincaré) yielded the highest accuracy.
  • Impact on Small Data: The most significant improvement was seen in the "Catalog Gifts & Tickets" genre—the smallest dataset—where the F-score increased by 19.7% over the baseline. This demonstrates that label-guided learning is exceptionally effective for "low-resource" classification where training examples are limited.

Future Directions

While successful, the authors identified several paths for further research. Currently, while label embeddings are initialized in hyperbolic space, the subsequent neural network training remains in Euclidean space; a fully hyperbolic network could potentially unlock further gains. Additionally, integrating these techniques with modern Transformer-based models like BERT and utilizing full label descriptions (rather than just names) could provide even richer semantic context for categorization.

Summary updated

BibTeX
@inproceedings{chen2021label,
  author = {Lei Chen and Hirokazu Miyake},
  title = {{Label-Guided Learning for Item Categorization in e-Commerce}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2021},
  doi = {10.18653/v1/2021.naacl-industry.37},
}
Figure