Sentiment and position-taking analysis of parliamentary debates: a systematic literature review

Gavin Abercrombie, Riza Batista-Navarro

Introduction

Parliamentary transcripts represent a vast repository of information regarding the policy preferences, attitudes, and ideological leanings of elected officials. Despite their value, research into this data is currently fragmented across two primary fields: Natural Language Processing (NLP) and Political/Social Science. This systematic review of 61 studies highlights a significant research "silo," where computer scientists focus on predictive tasks like vote estimation, while political scientists aim to map broad ideological shifts. Bridging this gap is essential for unlocking the full potential of legislative text analysis.

The Interdisciplinary Challenge

The authors identify three core challenges hindering progress in the field:

  1. Inconsistent Terminology: Concepts like "sentiment analysis," "ideal point estimation," and "political scaling" are often used interchangeably or with conflicting definitions across disciplines.
  2. Divergent Research Goals: Computer scientists typically treat transcripts as a classification problem (e.g., predicting a binary vote), whereas political scientists use them to characterize entire populations or measure institutional polarization.
  3. Methodological Gaps: Political science often relies on traditional "bag-of-words" models, missing out on modern NLP advancements. Conversely, NLP researchers frequently overlook the rich theoretical frameworks and expert-coded datasets provided by social scientists.

Methodology

To address these issues, the authors conducted a systematic literature review following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. They queried five major digital libraries, including the ACL Anthology and dblp, using a combination of sentiment analysis and legislative keywords.

PRISMA flow diagram of the systematic literature review process

From an initial pool of 1,043 publications, the field was narrowed to 61 empirical studies. These papers were then manually coded based on several dimensions, including research background, granularity of analysis, and technical approaches.

Framework for coding and analyzing the included publications

Data Landscape

The review highlights several benchmark datasets, though it notes a significant bias toward English-language data and the US Congress (featured in 31 of the 61 studies). Key sources include:

  • ConVote: A standard benchmark for US Congressional floor debates.
  • Hansard: Transcripts from the UK Parliament.
  • Talk of Norway: A richly annotated Norwegian corpus.
  • Europarl: Debates from the European Union Parliament.

Key Insights and Technical Results

The Complexity of Parliamentary Language

Detecting sentiment in a legislative context is uniquely difficult due to "polite parliamentary jargon." Formal, positive-sounding language is often used to mask scathing critiques, rendering traditional sentiment lexicons less effective. Furthermore, speaker metadata (such as party affiliation) often proves more predictive of a position than the actual text of the speech.

Methodological Performance

  • Algorithms: While Support Vector Machines (SVMs) remain a popular baseline, Neural Networks (such as LSTMs and CNNs) consistently demonstrate superior performance in direct comparisons.
  • Scaling Validity: Computational "scaling" methods (placing politicians on a spectrum) have been successfully validated against expert human surveys, confirming "text as data" as a legitimate social science tool.
  • Feature Engineering: Moving beyond "bag-of-words" to incorporate word embeddings and dependency parsing significantly improves the accuracy of identifying specific opinion targets.

Future Directions

The field faces several hurdles that require future attention:

  • Domain Adaptation: There is a lack of research on whether models trained on one legislature (e.g., US Congress) can generalize to others (e.g., UK Parliament).
  • Temporal Analysis: Few studies exploit the longitudinal nature of parliamentary data to track how political language and sentiment evolve over decades.
  • Granularity: Most current systems analyze whole speeches. Future work should move toward "argument mining" to extract sentiment toward specific policies mentioned within a speech.
  • Civic Application: There is a notable gap between academic research and the development of real-world "civic tech" tools that help citizens understand their representatives' positions.

Summary updated

Abstract

Parliamentary and legislative debate transcripts provide access to information concerning the opinions, positions, and policy preferences of elected politicians. They attract attention from researchers from a wide variety of backgrounds, from political and social sciences to computer science. As a result, the problem of computational sentiment and position-taking analysis has been tackled from different perspectives, using varying approaches and methods, and with relatively little collaboration or cross-pollination of ideas. The existing research is scattered across publications from various fields and venues. In this article, we present the results of a systematic literature review of 61 studies, all of which address the automatic analysis of the sentiment and opinions expressed, and the positions taken by speakers in parliamentary (and other legislative) debates. In this review, we discuss the existing research with regard to the aims and objectives of the researchers who work in this area, the automatic analysis tasks which they undertake, and the approaches and methods which they use. We conclude by summarizing their findings, discussing the challenges of applying computational analysis to parliamentary debates, and suggesting possible avenues for further research.
BibTeX
@article{abercrombie2020sentiment,
  author = {Gavin Abercrombie and Riza Batista-Navarro},
  title = {{Sentiment and position-taking analysis of parliamentary debates: a systematic literature review}},
  journal = {J. Comput. Soc. Sci.},
  year = {2020},
  doi = {10.1007/s42001-019-00060-w},
}
Figure