Measure Country-Level Socio-Economic Indicators with Streaming News: An Empirical Study

Bonan Min, Xiaoxi ZhaoEMNLP 2019 · DOI

TL;DR

This paper introduces Event-Centric Indicator Measure (ECIM), a novel method to track socio-economic indicators by analyzing events reported in streaming news. ECIM demonstrates strong correlations with established indicators like unemployment rate, VIX, and economic policy uncertainty, offering a new, data-driven approach to understanding economic health.

What Problem Does This Paper Address?

Measuring socio-economic conditions is crucial for policymakers and economists, but traditional methods like large-scale surveys are costly, time-consuming, and can struggle to capture nuanced indicators like economic uncertainty. The authors propose using the vast and continuously updated stream of news as a source of "micro-sensors" to infer these conditions. They aim to bridge the gap between the difficulty of direct measurement and the richness of real-time information available in news feeds.

Methods

The core of the ECIM approach involves a multi-step process:

  1. Event Extraction: The system first extracts "event mentions" from news text. This is achieved by leveraging predicate-argument structures derived from syntactic-semantic representations, specifically Semantic Role Labeling (SRL). Predicates (verbs and eventive nouns) are identified as event triggers, and their associated entities and temporal expressions are used to determine the event's location and time. Inference rules are applied to enrich location and time information when it's not directly adjacent to the trigger.
  2. Relevant Event Trigger Identification: For a given socio-economic indicator, the authors identify a set of relevant event trigger words or phrases. This involves processing a large corpus to extract event triggers, manual annotation to select those relevant to the target indicator, and then expanding the list using WordNet synsets and word embedding similarities.
  3. ECIM Calculation:
    • Filtering and Aggregation: Events are filtered by the target country. Then, for each time step (monthly in this study), the system counts the occurrences of relevant event triggers.
  4. Normalization: These raw counts are normalized by the total number of articles published in that month to account for variations in media activity.
  5. Smoothing: A moving average (7-time step window) is applied to the normalized counts to reduce noise and reveal underlying trends.

The final ECIM value for an indicator at a given time is the smoothed, normalized count of relevant events.

Datasets

  • English Gigaword Corpus: A large corpus of 5.7 million news articles from 1994 to 2010, used for training the event extraction system and generating ECIMs. Over 10 million event mentions were extracted from this corpus.
  • Publicly Available Socio-Economic Indicators:
    • Unemployment Rate (from the U.S. Bureau of Labor Statistics)
    • Chicago Board Options Exchange (CBOE) Volatility Index (VIX)
    • Economic Policy Uncertainty (EPU) index (Baker et al., 2016)

Key Insights and Ideas

  • News as "Micro-Sensors": The fundamental idea that streaming news can act as a distributed, real-time sensor network for socio-economic conditions is a powerful concept.
  • Event-Centric Approach: Focusing on specific events (e.g., "layoffs," "bankruptcies," "crises") rather than just keywords or sentiment provides a more objective and granular measure of economic phenomena.
  • Correlation with Established Indicators: The strong empirical correlation between ECIM and well-established indicators validates the approach and suggests its potential as a complementary or alternative measurement tool.
  • Capturing Lagged Effects: The observation that ECIM for unemployment rate sometimes shows a slight lead or matches the trend with a delay, mirroring real-world economic responses, is insightful.
  • Negative Correlation for Deeper Insights: The exploration of negative correlations, such as between "competition" events and unemployment, opens up avenues for measuring indicators when they are low, offering a more nuanced understanding.

Key Results

  • Strong Correlations: ECIM values showed strong correlations with the unemployment rate (Pearson correlation coefficient of 0.4286), EPU News (0.5136), and VIX (0.4115). All correlations were statistically significant (p-value < 0.0000).
  • Visual Agreement: Time series plots (Figure 3) visually demonstrate a high degree of agreement between ECIM trends and the actual socio-economic indicators, including peaks and general directional movements.
  • Negative Correlation Success: The study found a strikingly high negative correlation between "Competition" events (measured by 1/Mcomp) and the unemployment rate, suggesting the potential to measure indicators by events that occur when the indicator is low.

Limitations and Future Work

  • Event Extraction Sophistication: While SRL is used, the authors acknowledge that more sophisticated event extraction techniques could further improve accuracy.
  • Time Lags: Future work will focus on further studying the correlation with time lags, implying a need to better model the temporal dynamics between news events and economic outcomes.
  • Data Coverage: The study notes a slight deviation in EPU in 2003-2004 due to low coverage of the Iraq War in the Gigaword corpus, highlighting the dependence on the news source's breadth.
  • Indicator Scope: The current study focuses on three indicators; expanding to a wider range of socio-economic indicators is a natural next step.
  • Event Trigger Identification: The manual annotation step for identifying relevant event triggers, while efficient, could be further automated or refined.

Summary updated

BibTeX
@inproceedings{min2019measure,
  author = {Bonan Min and Xiaoxi Zhao},
  title = {{Measure Country-Level Socio-Economic Indicators with Streaming News: An Empirical Study}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2019},
  doi = {10.18653/v1/d19-1121},
}
Figure