The Sensitivity of Word Embeddings-based Author Detection Models to Semantic-preserving Adversarial Perturbations

Jeremiah Duncan, Fabian Fallas, Chris Gropp, Emily Herron, Maria Mahbub, Paula Olaya, Eduardo Lazcano‐Ponce, Tabitha Samuel, Daniel C. Schultz, Sudarshan Srinivasan, Maofeng Tang, В. И. Зенков, Quan Zhou, Edmon Begoli

Abstract

Authorship analysis is an important subject in the field of natural language processing. It allows the detection of the most likely writer of articles, news, books, or messages. This technique has multiple uses in tasks related to authorship attribution, detection of plagiarism, style analysis, sources of misinformation, etc. The focus of this paper is to explore the limitations and sensitiveness of established approaches to adversarial manipulations of inputs. To this end, and using those established techniques, we first developed an experimental framework for author detection and input perturbations. Next, we experimentally evaluated the performance of the authorship detection model to a collection of semantic-preserving adversarial perturbations of input narratives. Finally, we compare and analyze the effects of different perturbation strategies, input and model configurations, and the effects of these on the author detection model.

Introduction

In the era of digital misinformation and plagiarism, identifying the true author of a text has become a critical security necessity. Modern authorship attribution systems frequently utilize word embeddings—mathematical representations of words—to create a stylistic "fingerprint" of a writer. However, these models are potentially vulnerable to adversarial attacks. If an adversary understands the model's underlying mechanics, they can subtly manipulate their writing (e.g., swapping synonyms or altering contractions) to evade detection while preserving the original meaning. This research explores the sensitivity of established NLP approaches to such semantic-preserving changes, investigating whether these models capture deep authorial style or merely surface-level patterns.

Methodology

The researchers developed a framework termed author2vec, which adapts standard NLP techniques specifically for authorship tasks. The pipeline consists of three primary stages:

  1. Data Cleaning: Unlike standard NLP tasks that discard "stop words" (e.g., "the", "and"), this framework preserves them, as they are essential indicators of individual style.
  2. Author-Specific Embeddings: Instead of relying on generic pre-trained models, word2vec models were trained from scratch for each author to capture unique semantic nuances.
  3. Classification: Text is partitioned into documents of varying sizes (350, 1,400, or 3,500 words) and passed through a doc2vec model before being classified by a Multi-Layer Perceptron (MLP).
The author2vec experimental framework pipeline, illustrating the flow from data cleaning to final classification.

To test robustness, four types of adversarial perturbations were applied: synonym replacement (using NLTK Synsets), contraction expansion and pronoun shifts, dialect translation (US to UK English), and numerical transformations (digits to words).

Experimental Setup

The study utilized a corpus of approximately 300,000 words per author sourced from Project Gutenberg. To maintain genre consistency, the researchers selected three iconic mystery novelists: Agatha Christie, Sir Arthur Conan Doyle, and Mary Roberts Rinehart. The data was split 90/10 for training and testing, with the training set further partitioned for validation and hyperparameter optimization.

Analysis of Results

The experimental results revealed several key insights into the nature of automated authorship detection:

The Context Threshold

A significant finding was the relationship between document size and accuracy. As shown in the validation data, identifying an author from a small 350-word "page" is considerably more challenging than from a 3,500-word "short story." Accuracy improved dramatically as the partition size increased, suggesting that stylistic patterns require a certain volume of text to become statistically significant.

Mean validation accuracies across different document partition sizes (350, 1,400, and 3,500 words) and vocabulary sizes.

Resilience and Vulnerability

The models demonstrated remarkable robustness against vocabulary-based shifts. Even when 20% of the text was altered via synonym replacement or dialect translation, accuracy remained high (often between 0.95 and 1.0). This suggests that the models lean heavily on the "structural glue" of the language—the stop words and sentence constructions—rather than just specific nouns or adjectives.

However, a notable vulnerability was discovered regarding functional language. The most successful adversarial attack involved changing first-person pronouns and expanding contractions. For instance, in certain configurations for Agatha Christie, accuracy dropped from 0.90 to 0.71. This indicates that the models are highly tuned to how an author handles self-reference and informal phrasing.

Conclusion

While word-embedding-based models are resilient to simple thesaurus-based attacks, they exhibit sensitivity to shifts in functional and structural language. Future work should explore deeper architectures, such as Convolutional Neural Networks (CNNs) or Transformer-based models like BERT, which may offer enhanced robustness. Additionally, investigating more complex, manual perturbations—such as altering narrative tense or point-of-view—will be essential to understanding the full scope of adversarial threats in authorship analysis.

Summary updated

Abstract

Authorship analysis is an important subject in the field of natural language processing. It allows the detection of the most likely writer of articles, news, books, or messages. This technique has multiple uses in tasks related to authorship attribution, detection of plagiarism, style analysis, sources of misinformation, etc. The focus of this paper is to explore the limitations and sensitiveness of established approaches to adversarial manipulations of inputs. To this end, and using those established techniques, we first developed an experimental frame-work for author detection and input perturbations. Next, we experimentally evaluated the performance of the authorship detection model to a collection of semantic-preserving adversarial perturbations of input narratives. Finally, we compare and analyze the effects of different perturbation strategies, input and model configurations, and the effects of these on the author detection model.
BibTeX
@misc{duncan2021sensitivity,
  author = {Jeremiah Duncan and Fabian Fallas and Chris Gropp and Emily Herron and Maria Mahbub and Paula Olaya and Eduardo Lazcano‐Ponce and Tabitha Samuel and Daniel C. Schultz and Sudarshan Srinivasan and Maofeng Tang and В. И. Зенков and Quan Zhou and Edmon Begoli},
  title = {{The Sensitivity of Word Embeddings-based Author Detection Models to Semantic-preserving Adversarial Perturbations}},
  year = {2021},
  eprint = {2102.11917},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2102.11917},
}

Loading related content...

Figure