Company classification using zero-shot learning

Maryan Rizinski, Andrej Jankov, Vignesh Sankaradas, Eugene Pinsky, Igor Mishkovski, Dimitar TrajanovarXiv 2023 · arXiv · OpenReview

TL;DR

This paper introduces a zero-shot learning approach for classifying companies into industries using pre-trained NLP models and company descriptions. By leveraging transformer models without task-specific fine-tuning, it offers a more efficient and potentially automated alternative to traditional, labor-intensive industry classification standards.

What Problem Does This Paper Address?

Classifying companies into distinct industries is crucial for financial research, investment analysis, and market segmentation. Traditional methods rely on established standards like SIC, NAICS, and GICS. However, these systems suffer from several drawbacks: they are often difficult to interoperate, require significant human expert input (making them subjective and time-consuming to create and update), and struggle to keep pace with the dynamic nature of modern businesses and services. This leads to inconsistencies, delays, and high costs, highlighting a need for more automated and adaptable classification solutions.

Methods

The core of this paper's approach is zero-shot learning applied to company classification. They utilize pre-trained transformer models, specifically valhalla/distilbart-mnli-12-3, to extract meaningful features from textual company descriptions. Instead of fine-tuning these models on a labeled dataset for each specific industry category, the models are used "out-of-the-box" to classify companies into predefined categories. This means the model can classify companies into categories it hasn't been explicitly trained on, relying on its general understanding of language and semantic relationships.

To potentially improve performance, the authors experimented with modifying the industry category names. They used TF-IDF to identify important terms within each sector and created more descriptive labels. This was done to provide the zero-shot model with clearer signals for classification.

Datasets

The primary dataset used for evaluation is derived from the Wharton Research Data Services (WRDS). This dataset includes textual descriptions and GICS classifications for 34,338 publicly traded companies. The authors filtered the original Compustat dataset to ensure GICS sector assignments were present.

Key Insights and Ideas

  • Zero-shot learning as a viable alternative: The paper demonstrates that powerful pre-trained NLP models can perform company classification without task-specific fine-tuning, significantly reducing the need for labeled data and manual effort.
  • Transformer models' potential for industry understanding: Transformer architectures, with their ability to capture long-range dependencies in text, are well-suited for understanding the nuances of company descriptions and mapping them to industry categories.
  • The impact of category naming: Modifying industry category names, even with simple TF-IDF-based enhancements, can lead to noticeable improvements in classification performance, suggesting that clear and descriptive labels are crucial for zero-shot models.
  • Limitations of traditional standards: The paper reinforces the well-known issues with existing industry classification systems, such as lack of interoperability and slow adaptation to market changes, underscoring the value of automated approaches.

Key Results

The paper reports an initial weighted F1 score of 0.56 using the original GICS sector names. After modifying the sector names using TF-IDF and excluding certain words (like verbs and country names), the weighted F1 score improved to 0.64.

The classification performance varied significantly across industries. The best-performing categories included "Oil & Natural Gas, Consumable Fuels and Petroleum" (F1=0.81) and "Health Care" (F1=0.84) with the enhanced sector names. Conversely, categories like "Food, Beverages and Household Products" (F1=0.30), "Real Estate Properties" (F1=0.39), and "Industrials and Transportation" (F1=0.39) showed lower performance.

Limitations and Future Work

  • Performance on specific sectors: The authors acknowledge that the zero-shot approach still struggles with certain sectors, particularly those with lower F1 scores like Real Estate, Consumer Staples, and Industrials.
  • Need for further refinement: While the modified sector names improved performance, the paper suggests that further research into optimal label engineering or more sophisticated fine-tuning strategies could yield even better results.
  • Potential for unsupervised learning: The authors mention that their zero-shot approach can serve as a basis for future research in unsupervised learning for company classification.
  • Integration of additional data: The paper hints at the possibility of augmenting textual descriptions with other financial data (like PER or PBR) to enhance classification accuracy.

Summary updated

Abstract

In recent years, natural language processing (NLP) has become increasingly important in a variety of business applications, including sentiment analysis, text classification, and named entity recognition. In this paper, we propose an approach for company classification using NLP and zero-shot learning. Our method utilizes pre-trained transformer models to extract features from company descriptions, and then applies zero-shot learning to classify companies into relevant categories without the need for specific training data for each category. We evaluate our approach on a dataset obtained through the Wharton Research Data Services (WRDS), which comprises textual descriptions of publicly traded companies. We demonstrate that the approach can streamline the process of company classification, thereby reducing the time and resources required in traditional approaches such as the Global Industry Classification Standard (GICS). The results show that this method has potential for automation of company classification, making it a promising avenue for future research in this area.
BibTeX
@misc{rizinski2023company,
  author = {Maryan Rizinski and Andrej Jankov and Vignesh Sankaradas and Eugene Pinsky and Igor Mishkovski and Dimitar Trajanov},
  title = {{Company classification using zero-shot learning}},
  year = {2023},
  eprint = {2305.01028},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2305.01028},
}
Figure