A Corpus of Corporate Annual and Social Responsibility Reports: 280 Million Tokens of Balanced Organizational Writing

Sebastian G.m. Händschke, Sven Buechel, Jan Goldenstein, Philipp Poschmann, Tinghui Duan, Peter Walgenbach, Udo HahnACL 2018

TL;DR

This paper introduces JOCO, a massive new corpus of 280 million tokens from corporate annual and social responsibility reports, designed to fuel NLP research in economics and business. They demonstrate its utility by showing how linguistic "emotions" in these reports correlate with company performance, suggesting a link between corporate character and financial outcomes.

What Problem Does This Paper Address?

Natural Language Processing (NLP) research often relies on large, general-purpose text corpora. However, specialized domains like economics and business have unique linguistic characteristics (sublanguages) that can lead to performance degradation when using general corpora. While existing specialized corpora exist, they often focus on specific genres (like newspaper articles) or are outdated. This leaves a gap for a comprehensive, up-to-date corpus that captures the direct voice of economic actors – corporations themselves – in their own reporting. Such a corpus is crucial for developing NLP tools that can accurately analyze and understand economic and business phenomena.

Methods

The authors introduce the JOCO corpus, a collection of corporate annual reports (ARs) and corporate social responsibility reports (CSRRs) from top US, UK, and German companies. They collected approximately 5,000 reports spanning from 2000 to 2015, totaling 282 million tokens.

For data processing, they used:

  • PDF to Text Conversion: Standard tools were employed to extract raw text.
  • Rule-Based Refinement: Iterative rule-based methods were used to clean the extracted text, restoring structure, removing artifacts, and handling hyphenation.
  • NLTK for Corpus Analysis: Standard NLP tools were used for token and sentence counting.
  • Word Embeddings: FastText was used to train word embeddings on the corpus, capturing the semantic nuances of economic jargon. They focused on words with a frequency of at least 50 and excluded subword information to mitigate PDF conversion artifacts.
  • Regression Analysis: For their experimental analysis, they employed Generalized Estimating Equations (GEE), a time-series model suitable for repeated observations, to analyze the relationship between linguistic features and company performance (Return on Equity - ROE). They used a tool called JEMAS to extract linguistic features related to emotions (Valence, Arousal, Dominance) from the reports.

Datasets

The primary dataset introduced in this paper is the JOCO corpus.

  • Composition: 280 million tokens from approximately 5,000 corporate annual reports (ARs) and corporate social responsibility reports (CSRRs).
  • Scope: Reports from top US, UK, and German companies across major, mid-sized, and technology stock indices (DJIA, S&P 500, NASDAQ; FTSE 100, FTSE 250, FTSE AIM 100; DAX, MDAX, TECDAX).
  • Timeframe: 2000-2015.
  • Comparison to Existing: The authors note that the JOCO corpus is smaller in terms of tokens and companies than the influential 10-K Corpus (Kogan et al., 2009) but includes CSRRs, covers a more recent time span (up to 2015 vs. 2006), and offers a broader international perspective.

Key Insights and Ideas

  • Direct Corporate Voice is Valuable: By analyzing ARs and CSRRs, researchers can access the unfiltered communication of economic actors, offering a more direct perspective than media interpretations.
  • Emotional Language Matters for Performance: The study suggests that the way companies express "emotions" (measured by Arousal and Dominance) in their reports has a tangible impact on their financial performance (ROE).
  • Calm Communication Correlates with Better Performance: Higher "Arousal" (indicating more intense emotional expression) in reports was linked to lower ROE, suggesting that calmer, more measured communication might be associated with better financial outcomes.
  • Country-Specific Nuances Exist: The interaction between emotional language and performance varies by country, indicating that cultural or market-specific factors influence how corporate communication translates into financial results.
  • JOCO Enables Cross-Comparisons: The corpus's balanced selection across indices, industries, and countries makes it ideal for deeper economic investigations that require cross-sectional analysis.

Key Results

  • The JOCO corpus contains 282 million tokens from nearly 5,000 reports from 270 companies.
  • In their experimental analysis, they found that Arousal had a significant (p < .001) negative effect on ROE, meaning companies with calmer communication performed better.
  • This effect was more pronounced for British companies compared to German companies, as indicated by a significant interaction term.
  • The word embeddings generated from JOCO showed high face validity, with terms like "growth" correctly associating with economic contexts rather than biological ones.

Limitations and Future Work

  • Data Acquisition Challenges: The authors note that acquiring older reports, especially CSRRs from the early 2000s, was challenging.
  • IPR Restrictions: The plain text data of JOCO is restricted by Intellectual Property Rights (IPR), necessitating the use of word embeddings as a publicly shareable alternative.
  • Subsample for Experiment: The experimental analysis on organizational emotions was conducted on a subsample of British and German firms from 2008-2015, not the entire corpus.
  • Future Work: The paper suggests that the JOCO corpus is well-suited for a wide range of NLP applications in economics, business, and management research, implying future work could explore various analytical tasks and models. The availability of the trained embedding model also opens doors for further research.

Summary updated

BibTeX
@inproceedings{handschke2018corpus,
  author = {Sebastian G.M. Händschke and Sven Buechel and Jan Goldenstein and Philipp Poschmann and Tinghui Duan and Peter Walgenbach and Udo Hahn},
  title = {{A Corpus of Corporate Annual and Social Responsibility Reports: 280 Million Tokens of Balanced Organizational Writing}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2018},
}
Figure