Multi-Task Learning for Authorship Attribution via Topic Approximation and Competitive Attention

Wei Song, Chen Zhao, Lizhen LiuIEEE Access 2019 · DOI

Introduction

Authorship Attribution (AA) is the process of identifying the creator of a text from a set of candidate authors. While deep learning has significantly advanced this field, models often struggle with "topic bias"—the tendency to identify authors based on the subjects they write about rather than their unique writing style. This paper presents a novel neural multi-task learning (MTL) framework designed to disentangle stylistic markers from thematic content.

The Challenge of Topic Bias

In traditional AA, topic and style are deeply intertwined. If an author's training data consists primarily of political commentary, a standard model might incorrectly learn that political keywords are indicators of that author's style. Consequently, if the author writes about a different topic, such as sports, the model's accuracy drops. Previous solutions, like manually removing content words, often discard valuable stylistic information.

Conceptual overview of separating topic and style representations for authorship attribution.

Methodology

The proposed framework utilizes three innovative components to achieve robust topic-style separation:

1. Topic Approximation

To avoid the need for manual topic labeling, the authors employ an unsupervised Latent Dirichlet Allocation (LDA) model trained on a large external corpus. The neural network includes an auxiliary task where the "topic representation" is trained to approximate these LDA-generated distributions, providing a probabilistic guide for identifying thematic content.

2. Competitive Attention Mechanism

The core of the architecture is a competitive attention mechanism. Unlike standard attention, this approach forces topic and style representations to compete for the attention of each word. Words with high thematic relevance (e.g., "technology") are assigned lower weights in the style representation, while stylistic markers (e.g., specific adverbs or prepositions) are prioritized for the authorship task. This handles "swing words"—words that can serve as either topic or style markers depending on context—more effectively than static filters.

3. Separation-Reconstruction Constraint

To ensure the separated vectors retain the original text's semantic essence, the model incorporates a reconstruction constraint. This component attempts to rebuild the original shared text representation from the individual topic and style components, minimizing the loss between the original and the reconstructed versions.

The separation-reconstruction framework ensuring representation integrity.

Experimental Results

The model was evaluated on the IMDb62 dataset and a newly introduced Novel8 dataset, which includes works by eight famous novelists.

  • Cross-Topic Performance: On the Novel8 "Heterologous" set (where training and testing topics differ entirely), the model achieved 71.38% accuracy, significantly outperforming baseline models like CNN-CharNgrams (60.64%).
  • Robustness to Topic Leakage: On a specialized "IMDb62-Hard" subset designed to maximize topic bias, the model maintained 91.51% accuracy, demonstrating its ability to ignore misleading content keywords.
  • Interpretability: Analysis of attention weights confirmed that the model successfully ignores content-heavy words (like "music" or "show") in favor of stylistic nuances when performing attribution.

Conclusion

By treating style as the "residual" signal left after accounting for topic, this framework provides a more robust and interpretable approach to authorship attribution. The competitive attention mechanism and reconstruction constraints allow the model to excel in challenging cross-topic scenarios where traditional methods often fail. Future work will explore more advanced topic models and the application of this competitive framework to other text-separation tasks.

Summary updated

BibTeX
@article{song2019multi,
  author = {Wei Song and Chen Zhao and Lizhen Liu},
  title = {{Multi-Task Learning for Authorship Attribution via Topic Approximation and Competitive Attention}},
  journal = {IEEE Access},
  year = {2019},
  doi = {10.1109/access.2019.2957152},
}
Figure