An Empirical Study on Academic Commentary and Its Implications on Reading and Writing

Tai Wang, Xiangen Hu, Keith T. Shubeck, Zhiqiang Cai, Jie TangarXiv 2016 · arXiv

Abstract

The Reading and Writing Relationship (RRW) is a foundational concept in educational psychology, yet quantifying the influence of a writer's latent background knowledge remains a significant methodological hurdle. This study proposes a novel approach using academic commentaries as a "natural laboratory." By analyzing citation lists as proxies for background knowledge and applying Latent Dirichlet Allocation (LDA), we developed a Topic Process Model to measure how existing knowledge modulates the response to a source text. Our analysis of a scholarly debate on video game addiction reveals the phenomenon of "bibliographic resonance," where the degree of thematic overlap between authors is fundamentally tied to their shared—or disparate—citation histories.

Introduction

A central challenge in learning sciences is measuring how a reader's prior knowledge interacts with a text to produce new writing. Traditional pre-tests often introduce confounding variables. To address this, we utilize the genre of academic commentary. In this framework, the "input" (the original paper) and "output" (the commentary) are explicitly linked, and the author's mandatory citations provide a verifiable map of their background knowledge.

Conceptual model of academic commentary as a research framework for reading and writing.

Methodology

The study evaluates the RRW through citation vector analysis and topic modeling.

The Topic Process Model

We define the formation of output topics ($O_k$) using a model inspired by Hebbian learning: $$O_k = B_k + \gamma_k B_k I_k$$ Where $B_k$ represents background knowledge (derived from the author's citations), $I_k$ is the input from the original paper, and $\gamma_k$ is a coefficient representing the interaction between the two.

Case Study: Video Game Addiction Debate

The empirical analysis focused on a debate surrounding video game addiction, initiated by an original paper by Wood (2008). We analyzed three commentaries representing different standpoints (pros and cons) to observe how their background knowledge influenced their critique.

Summary of the papers, standpoints, and citation counts used in the case study.

Key Results

Bibliographic Resonance

Our findings show that when authors agree (e.g., Author B and Author W), their citation vectors exhibit significant overlap. Conversely, when authors disagree (e.g., Authors G/T and Author W), there is nearly zero overlap in their references. This suggests that scientific disagreement is often a result of authors operating within entirely different "knowledge universes," a phenomenon we term bibliographic resonance.

Topic Modulation

The LDA results demonstrate that writing is a process of modulation. Author B, who supported the original paper, showed a high correlation (0.9600) between input and output topics. However, Author G, who strongly disagreed, showed a correlation of -0.0124, indicating the original paper’s topics had almost no influence on their response. In cases of disagreement, the output paper's thematic structure followed the author's background knowledge far more closely than the input text.

Conclusion

This study provides a mathematical basis for understanding how background knowledge shapes the writing process. While the current analysis is limited to a small sample of academic authors, it offers a robust framework for future psychometric research. Moving forward, we aim to apply word-level analysis to capture finer nuances in the interaction between reading and writing.

Summary updated

BibTeX
@misc{wang2016empirical,
  author = {Tai Wang and Xiangen Hu and Keith T. Shubeck and Zhiqiang Cai and Jie Tang},
  title = {{An Empirical Study on Academic Commentary and Its Implications on Reading and Writing}},
  year = {2016},
  eprint = {1602.04101},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1602.04101},
}
Figure