Generating Domain-Specific Dictionaries Using Bayesian Learning

Nicolas Prrllochs, Stefan Feuerriegel, Dirk Neumann

TL;DR

This paper tackles the challenge of using sentiment analysis for financial news by creating domain-specific dictionaries. Instead of relying on manually curated word lists, they employ Bayesian variable selection methods (Ridge, LASSO, Elastic Net) to statistically identify words that truly influence investor decisions, leading to significantly improved predictive performance compared to existing methods.

What Problem Does This Paper Address?

Financial news is a crucial source of information for investors, but its subjective nature makes sentiment analysis difficult. Traditional approaches often use general-purpose dictionaries (like Harvard-IV) or manually curated lists for specific domains. However, these methods have limitations: words can have different connotations in a financial context, and manually selected words are assumed to have equal importance, which might not be true. This paper aims to overcome these limitations by automatically generating domain-specific dictionaries for financial news that capture the nuanced sentiment relevant to investors.

Methods

The core of this paper lies in using Bayesian variable selection techniques to generate domain-specific dictionaries. The process involves:

  1. Preprocessing: Cleaning text, removing stop words, stemming words, and creating a document-term matrix. Crucially, they also apply TF-IDF weighting to give more importance to discriminative words.
  2. Dictionary Generation:
    • Ridge Regression: A regularization technique that shrinks coefficients towards zero but doesn't set them exactly to zero. From a Bayesian perspective, this corresponds to a Gaussian prior.
  3. LASSO (Least Absolute Shrinkage and Selection Operator): Another regularization technique that can shrink coefficients exactly to zero, effectively performing variable selection. This is modeled with a double-exponential (Laplace) prior.
  4. Elastic Net: A combination of Ridge and LASSO, offering a balance between shrinkage and selection. This is represented by a compromise between Gaussian and Laplacian priors.
  5. OLS (Ordinary Least Squares): Included as a baseline comparison.

The key novelty is treating every word as a potential regressor and using these Bayesian methods to statistically determine which words are truly influential in predicting stock market returns, thereby creating weighted dictionaries.

Datasets

  • Training/Dictionary Generation Data: German regulated ad hoc announcements from January 2004 to June 2011. The corpus consists of 14,463 announcements, each with at least 50 words and written in English.
  • Stock Market Data: Daily abnormal returns of corresponding companies, derived using event study methodology and a market model based on the CDAX index.
  • Validation Set: A subset of the ad hoc announcements from 2011-2012 (925 observations) used to evaluate the out-of-sample predictive performance of the generated dictionaries.

Key Insights and Ideas

  • Context is King: Words classified as positive in general dictionaries are not always interpreted positively in financial news. The interpretation is highly context-dependent.
  • Manual Selection is Flawed: Manually curated dictionaries can lead to misclassification due to domain-specific nuances (e.g., "adjustment" can have negative connotations in financial contexts).
  • Bayesian Regularization Outperforms: Bayesian variable selection methods (Ridge, LASSO) generate dictionaries that are superior to existing static dictionaries in terms of goodness-of-fit and predictive performance.
  • Ridge Regression Shines: Among the regularization methods, Ridge regression consistently performed best, suggesting that while variable selection is important, the shrinkage property is more critical for this domain.
  • Investors Recognize Framing: Investors are adept at recognizing when companies frame negative information using positive language.

Key Results

  • Predictive Performance: Ridge regression achieved the highest correlation with abnormal stock market returns on the validation set, showing an improvement of 93.25% compared to the best-performing existing dictionary (Harvard-IV).
  • Goodness-of-Fit: While OLS had the highest in-sample R² (0.0382), the regularization methods (Ridge: 0.0271, LASSO: 0.0199) offered better generalization by mitigating overfitting and multicollinearity.
  • Dictionary Similarity: The generated dictionaries showed higher similarity to the Loughran-McDonald (LM) dictionary than others, indicating a better fit for the financial domain.
  • Elastic Net Behavior: The elastic net's performance was maximized at the boundaries of its parameter range, effectively reducing to either Ridge (α=0) or LASSO (α=1), with Ridge yielding better results.

Limitations and Future Work

  • Limitations: The paper acknowledges that while regularization methods improve goodness-of-fit and predictive performance, they do lead to a lower in-sample R² compared to OLS. The elastic net's performance was best when it essentially became Ridge regression, suggesting that the l₁-norm penalty might not be as beneficial in this specific context.
  • Future Work:
    • Broader Data Sources: Validate the predictive performance of Bayesian variable selection on news from other financial domains.
  • Advanced Methods: Explore spike and slab regression and F-LASSO for enhanced variable selection.
  • Negation Scope Detection: Integrate variable selection with negation scope detection to better handle inverted meanings and reduce misclassification.

Summary updated

Abstract

This paper aims to operationalize subjective information processing in financial news disclosures. In order to measure news tone, previous research commonly utilizes manually-selected positive and negative word lists, such as the Harvard-IV psychological dictionary. However, such dictionaries may not be suitable for the domain of financial news because positive and negative entries could have different connotations in a financial context. To overcome the problem of words that are selected ex ante, we incorporate several Bayesian variable selection methods to select the relevant positive and negative words from financial news disclosures. These domain-specific dictionaries outperform existing dictionaries in terms of both their explanatory power and predictive performance, resulting in an improvement of up to 93.25% in the correlation between news sentiment and stock market returns. According to our findings, the interpretation of words strongly depends on the context and managers need to be cautious when framing negative content using positive words.
BibTeX
@inproceedings{prrllochs2015generating,
  author = {Nicolas Prrllochs and Stefan Feuerriegel and Dirk Neumann},
  title = {{Generating Domain-Specific Dictionaries Using Bayesian Learning}},
  booktitle = {ECIS},
  year = {2015},
  url = {https://openreview.net/forum?id=U38jNa1uIP_7},
}
Figure