Authorship Attribution Using Probabilistic Context-Free Grammars

Sindhu Raghavan, Adriana Kovashka, Raymond J. MooneyACL 2010 · OpenReview

Introduction

Authorship attribution—the task of identifying the author of a document based on writing style—is a critical tool in forensics, plagiarism detection, and historical research. Traditionally, this field has relied on surface-level "style markers" such as word frequencies, character n-grams, or part-of-speech distributions. However, these methods often overlook the deeper structural patterns of language. This paper proposes that an individual’s unique "syntactic fingerprint"—the way they construct sentences—is a powerful but underutilized signal for identity. By modeling an author’s style as a distinct "language" using Probabilistic Context-Free Grammars (PCFGs), the researchers demonstrate a novel approach to capturing these structural preferences.

Methodology

The core of the proposed system is a generative pipeline that transforms raw text into author-specific grammars:

  1. Generic Parsing: Because author-specific treebanks are unavailable, the researchers use the Stanford Parser (trained on generic corpora like the WSJ) to automatically generate parse trees for training documents.
  2. Author-Specific PCFGs: A unique PCFG ($G_i$) is trained for each author ($A_i$) based on these generated trees.
  3. Classification: For a new document, the model calculates the likelihood of its sentences under each author's grammar. The author with the highest likelihood is predicted.
  4. Smoothing and Ensembles: To handle data sparsity, the authors introduced PCFG-I (interpolated with a background corpus) and PCFG-E, an ensemble model that combines syntactic PCFG scores with lexical features from Maximum Entropy (MaxEnt) and n-gram classifiers.

Datasets and Experimental Setup

The method was evaluated across five diverse datasets to distinguish stylistic signals from topical ones. The news datasets (Football, Business, Travel, and Cricket) were curated to ensure multiple authors wrote on the same topics, preventing the model from simply performing topic identification. A Poetry dataset from Project Gutenberg provided a test for more creative and syntactically varied text.

Summary statistics for the five datasets used in the study, including word and document counts per author.

Key Results and Discussion

The experimental results highlight the complementary nature of syntactic and lexical features. The ensemble model, PCFG-E, achieved the highest accuracy on four out of five datasets, reaching a peak of 95% on the Cricket dataset.

Authorship attribution accuracy (%) across different models and datasets, highlighting the performance of PCFG variants and ensembles.

The Syntactic Signal

In the Football dataset, the standalone PCFG model achieved 93.34% accuracy, significantly outperforming the MaxEnt (84.45%) and Bigram (86.67%) baselines. This suggests that in certain genres, syntactic choices are more distinctive than word choices. Interestingly, smoothing (PCFG-I) actually hindered performance in the Football dataset, implying that these authors had such highly idiosyncratic styles that adding generic grammar data only served to dilute the signal.

Poetry and Creative Expression

The PCFG approach proved particularly effective for poetry. The authors hypothesize that the loose and varied syntactic structure of poetry—compared to the standardized "AP style" of news—makes an author's structural choices more salient. For this dataset, interpolation (PCFG-I) was vital, raising accuracy from 78.18% to 83.63% by compensating for the sparse data inherent in literary works.

Conclusion

By treating syntax as a "dialect" of an individual author, this research provides a robust framework for authorship attribution. While PCFGs are data-hungry and sensitive to parser bias, their integration into ensemble models provides a state-of-the-art solution that captures both "what" an author says and "how" they structure it. Future work will explore discriminative training to further refine the model's ability to distinguish between closely related writing styles.

Summary updated

BibTeX
@inproceedings{raghavan2010authorship,
  author = {Sindhu Raghavan and Adriana Kovashka and Raymond J. Mooney},
  title = {{Authorship Attribution Using Probabilistic Context-Free Grammars}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2010},
  url = {https://openreview.net/forum?id=VLz3xExxL7W},
}
Figure