LATEX-Numeric: Language Agnostic Text Attribute Extraction for Numeric Attributes

Kartik Mehta, Ioana Oprea, Nikhil RasiwasiaNAACL 2021 · DOI

TL;DR

This paper introduces LaTeX-Numeric, a novel framework for automatically extracting numeric attributes from e-commerce product descriptions. It tackles the challenge of "missing-PA" (partial annotations due to missing attribute values) in distant supervision by proposing a multi-task learning architecture and an automated alias creation method, significantly improving extraction accuracy.

What Problem Does This Paper Address?

E-commerce platforms are flooded with product information, much of which is unstructured text. Extracting structured product attributes (like "16GB" for RAM) from this text is crucial for product discovery, comparison, and recommendation systems. While attribute extraction is a known problem, existing methods often rely on manually curated training data, which is expensive and doesn't scale to the vastness of e-commerce.

Distant supervision, which uses existing structured data to automatically generate training labels, offers a scalable alternative. However, for numeric attributes, it faces unique challenges:

  • Partial Annotations (Missing-PA): If an attribute's value is missing in the structured data, even if present in the text, distant supervision creates incomplete or incorrect labels. This is a significant issue not well-addressed in prior work.
  • Diverse Surface Forms (Aliases): Numeric attributes can be expressed in many ways (e.g., "2 mp", "2 mpix", "2 megapixels" for resolution).
  • Confusing Attributes & Units: Similar units can refer to different attributes (e.g., "GB" for RAM vs. storage), and sellers use various units (e.g., kg vs. pounds).

LaTeX-Numeric aims to overcome these limitations to build high-precision, language-agnostic numeric attribute extraction models without manual intervention.

Methods

The core of LaTeX-Numeric is a two-pronged approach:

  1. Multi-Attribute Joint Extraction (MAMT-NER): To address the "Missing-PA" problem, the authors propose a multi-task learning architecture. Instead of training a separate model for each attribute, MAMT-NER treats each attribute extraction as a separate task within a single model. Crucially, the loss for tasks where the attribute value is missing is deactivated. This allows the model to learn from available data without being penalized by missing information, leading to improved F1 scores for both numeric and non-numeric attributes. This is a departure from single-task approaches or methods that simply exclude samples with missing values.
  2. Automated Alias Creation: To handle the diverse surface forms of numeric attributes, the paper introduces an automated algorithm. This algorithm leverages product text and existing attribute values to generate a comprehensive list of "aliases" (different ways units are expressed). It involves:
  3. Attribute-specific alias_dw: Using regex to find common surface forms of units directly following attribute values.
  4. Description-based alias_bp: Using regex to find potential units in product text and then filtering them using word embeddings (like GloVe) to match them with canonical units.
  5. Embedding-based filtering: Further refining alias_bp candidates by checking their similarity to canonical units in embedding space.
  6. Exclusive Alias Flag: Identifying attributes with unique aliases to improve matching. This "auto-aliasing" process significantly enhances the quality of training annotations derived from distant supervision.

The underlying model architecture used is a BiLSTM-CNN-CRF, a standard for sequence labeling tasks, but the framework's novelty lies in the MAMT-NER and auto-aliasing components.

Datasets

The paper uses real-world e-commerce data:

  • English Marketplaces: Data was collected from five product categories across three English marketplaces (IN, US, UK). This data was split into 80% for training (used for alias creation and annotation generation) and 20% for testing.
  • Romance Languages: To test language-agnosticism, a similar analysis was performed on one product category for three Romance languages: French (FR), Spanish (SP), and Italian (IT).

The authors do not explicitly mention creating new benchmark datasets but rather use their collected data for evaluation.

Key Insights and Ideas

  • Missing-PA is a critical challenge: The paper highlights that the "Missing-PA" problem, arising from missing attribute values in distant supervision, is a significant hurdle for numeric attribute extraction and has been largely overlooked.
  • Multi-task learning is more robust: Treating attribute extraction as a multi-task problem, where tasks with missing data are masked, is more effective than single-task approaches or simply discarding incomplete data.
  • Automation is key for scalability: Manually curating aliases and training data is infeasible for e-commerce. The automated alias creation method is crucial for building scalable and accurate extraction systems.
  • Language-agnosticism is achievable: The proposed methods demonstrate effectiveness across different languages, suggesting a generalizable approach.

Key Results

  • MAMT-NER Improvement: The multi-task architecture (MAMT-NER) achieved a 9.2% F1 improvement for numeric attributes and a 7.4% improvement for non-numeric attributes compared to a single-task architecture (MAST-NER).
  • Auto-aliasing Impact: The automated alias creation method led to a significant 20.2% F1 improvement for numeric attributes compared to simpler matching techniques like "exact match" and "canonical aliasing."
  • Language Agnostic Performance: For Romance languages, the combined approach (auto-aliasing + MAMT-NER) showed a 13.9% F1 improvement.
  • BERT Compatibility: The MAMT architecture also showed improvements when used with BERT as the underlying model, indicating its flexibility.

Limitations and Future Work

  • Manual Effort for Multiplicative Factors: While the automated alias creation is powerful, the paper notes that manually determining multiplicative factors for converting units (e.g., pounds to kg) could further improve training annotations. This is left as future work.
  • Memory Constraints for BERT: The authors encountered memory errors when training BERT models on larger datasets (US and UK English), limiting their full evaluation in that context.
  • Potential for Collisions: The regex-based matching for alias creation can lead to collisions (e.g., matching "5 ghz" when looking for "5 GB"). While they mitigate this by ignoring multiple matches, it's a potential area for refinement.
  • Future Work: The authors suggest exploring the manual effort for multiplicative factors and further investigating the application of their framework to textual attributes and other underlying models.

Summary updated

BibTeX
@inproceedings{mehta2021latex,
  author = {Kartik Mehta and Ioana Oprea and Nikhil Rasiwasia},
  title = {{LATEX-Numeric: Language Agnostic Text Attribute Extraction for Numeric Attributes}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2021},
  doi = {10.18653/v1/2021.naacl-industry.34},
}
Figure