Abstract
Detecting political bias in "expert" text, such as academic research, is challenging due to institutional norms of objectivity and the confounding influence of research topics. This study develops a supervised machine learning approach to identify latent ideology in economics papers. By utilizing a topic-adjusted ensemble model, the researchers successfully predicted the political leanings of economists with 69.2% accuracy. The findings suggest that even ostensibly neutral scientific discourse carries subtle ideological signatures that can be systematically identified.
Introduction
While political partisanship is easily identified in editorials or speeches, it remains largely hidden in expert domains. Academic economics, in particular, views itself as an objective science, employing peer review and prohibitions on explicit policy advocacy to maintain neutrality. However, critics suggest the field is deeply politicized. The challenge for natural language processing (NLP) is twofold: ideology in these texts is latent rather than explicit, and it is often confounded by the subject matter. For instance, a researcher's focus on "bank notes" versus "mental health" might reflect their field of specialization rather than their political stance.
Methodology
The researchers proposed a supervised ensemble n-gram model designed to explicitly adjust for these topic confounds.
Feature Extraction and Topic Mapping
The model utilizes n-grams (lengths 2 to 8) and "skipgrams" to capture nuanced phrasing. To address the topic confound, papers were mapped to subjects using two distinct methods:
- Supervised Mapping: Utilizing standard Journal of Economic Literature (JEL) classification codes.
- Unsupervised Mapping: Employing Latent Dirichlet Allocation (LDA) to automatically discover 30, 50, or 100 distinct topics.
The Algorithm
Within each identified topic, the model uses Partial Least Squares (PLS) regression to isolate phrases most correlated with ideology. To ensure robustness and prevent overfitting, an ensemble approach was used, involving 125 iterations of sub-sampling to create a consensus ideological score.
Data Sources
The study combined text corpora with "ground truth" labels:
- Ground Truth: 2,204 economists were labeled as "left-leaning" or "right-leaning" by linking the American Economics Association (AEA) directory to federal campaign contributions and public petition signatures.
- Text Corpus: A dataset of 17,503 NBER working papers and 62,888 JSTOR research articles (1973–2011).
- Validation: Predicted scores were compared against survey responses from the University of Chicago’s IGM Expert Panel.
Ideological Signatures in Economic Text
A key finding is that ideology in academic text is a "second-order" signal. While unsupervised models often fail because they prioritize the primary signal (the research topic), supervised models can "tune in" to the subtle ideological variations within those topics.
As shown in the table above, right-leaning economists are more likely to use terms like free bank or feder reserv, while left-leaning economists favor terms like child care and mental health. Without topic adjustment, a model might incorrectly label any health economist as liberal. By comparing word choices within the same topic, the model captures how different ideological perspectives describe the same economic phenomena.
Results and Performance
The inclusion of topic adjustment significantly improved model performance. The best-performing configuration, using LDA with 50 topics (LDA50), achieved an accuracy of 69.2% and an Area Under the Curve (AUC) of 0.719.
The model's predictions also generalized well beyond the training data. Predicted ideology scores from academic papers were highly correlated with real-world policy stances expressed in the IGM Expert Panel surveys, even after controlling for demographic factors such as gender, university affiliation, and career stage.
Conclusion and Future Directions
While the 69% accuracy rate is lower than that typically seen in models analyzing overtly political text like Congressional speeches, it represents a significant breakthrough for "neutral" expert text. The study demonstrates that academic jargon is not purely technical but carries underlying worldviews.
Future research may focus on reducing the dependency on supervised "ground truth" labels and exploring how these tools can help policymakers "de-bias" research by identifying the latent leanings of cited authors. Ultimately, this work provides a framework for understanding how ideology permeates even the most rigorous scientific disciplines.
Summary updated