Introduction
Computational text analysis currently faces a "descriptive" bottleneck. While researchers are increasingly adept at reporting what patterns exist within large-scale datasets, there is a growing deficit in explaining why these patterns occur. This position paper argues that for the field of computational social science to mature, it must pivot from mere pattern matching toward "integrative" research grounded in established social science theory.
The Problem: Methods Over Meaning
The "honeymoon period" of computational social science has seen a fruitful exchange between Natural Language Processing (NLP) and social science. However, a significant gap has emerged. Much of the current research is purely descriptive, utilizing complex models to catalog evidence—such as identifying common topics in social media discourse—without addressing the broader "So what?" question.
By prioritizing "state-of-the-art" performance and objective reductionism, computer science often overlooks the contextual and critical frameworks necessary for interpreting human behavior. Without theory, the field risks measuring the rate of a falling apple without ever conceptualizing gravity.
Methodology and Taxonomy
To evaluate the state of the field, the authors conducted a meta-analysis of approximately 60 papers published in major NLP venues (e.g., ACL, EMNLP, NAACL). They categorized these works into two primary buckets:
- Descriptive: Findings that uncover patterns (e.g., word clusters or censorship rates) but fail to link them to broader social or communication theories.
- Integrative: Research that offers causal explanations or tests specific theory-based hypotheses, enabling predictive insights.
The authors trace this "empirical turn" back to 1996, noting a historical shift in NLP from linguistic theory toward machine learning performance.
Analysis of the Field
The paper’s qualitative critique highlights a trend where engineering-focused goals often overshadow scientific inquiry.
Descriptive Research
Many high-profile papers provide rigorous data analysis but miss opportunities to ground their findings in theoretical frameworks. For instance, studies on polarization or media manipulation often omit explicit links to "propaganda theory" or "social conflict theory."
Integrative Research
Conversely, "gold standard" works demonstrate the power of theory-grounded analysis. A notable example is Nelson (2021), which utilized word embeddings to confirm specific features of identity theory within 19th-century narratives.
Key Insights: The "Two Masters" Problem
Interdisciplinary researchers often face the "Two Masters" dilemma: computer science reviewers demand methodological complexity, while social science reviewers require theoretical depth. This tension frequently results in "safe," descriptive papers that fail to satisfy either discipline fully. Theory should not be viewed as a constraint but as an empowering organizing principle that moves the field beyond "mathiness" toward a genuine understanding of social processes.
Conclusion and Future Directions
The authors urge the community to rebalance the pendulum by:
- Prioritizing research questions that are theory-affirming or theory-building.
- Embracing a wider variety of methods, including qualitative and ethnographic approaches, when they better suit the research question.
- Encouraging major NLP venues to value interdisciplinary depth over incremental methodological improvements.
By integrating social science theory into computational workflows, the field can move beyond describing the "fall" and start identifying the "gravity" of social phenomena.
Summary updated