NLP Security and Ethics, in the Wild

Heather C. Lent, Erick Galinkin, Yiyi Chen, Jens Myrup Pedersen, Leon Derczynski, Johannes Bjerva

Introduction

As Large Language Models (LLMs) transition from laboratory environments to real-world applications, the field of NLP Security (NLPSec) has seen unprecedented growth. However, this rapid expansion has occurred within an ethical "gray area." A systematic survey of 80 top-tier research papers reveals a significant culture gap: over half of the publications contain no discussion of ethics, none practiced coordinated vulnerability disclosure (CVD), and a heavy focus on English-centric models leaves speakers of low-resource languages at disproportionate risk.

The Ethical Landscape of NLPSec

The intersection of Natural Language Processing and Cybersecurity creates a unique "Devil’s Triangle." While traditional cybersecurity relies on established norms like Coordinated Vulnerability Disclosure (CVD) and harm minimization, the NLP community has historically prioritized transparency and open publication.

Hacker Alignment and the role of NLP Researchers

In NLPSec, publishing an exploit without prior warning to model creators—a common practice in NLP—can inadvertently empower malicious actors. The authors argue that much of current NLPSec research falls into "gray hat" territory: well-intentioned but lacking the accountability frameworks necessary for high-stakes security work. They propose a shift toward "White Hat NLP," where research is grounded in both cybersecurity best practices and NLP ethical norms.

Methodology

This meta-study analyzed 80 peer-reviewed papers from major venues (ACL, EMNLP, NAACL) published between 2019 and 2024. The authors manually annotated these works based on attack scenarios (adversarial, backdoor, data reconstruction), ethics discussions, awareness of dual-use/misuse, and adherence to CVD.

Key Findings: The Ethics Silence

The survey highlights a stark lack of ethical reflection within the community. Approximately 51% of the surveyed papers included no discussion of ethics, even when introducing novel, weaponizable attacks.

Prevalence of ethics discussions across different paper contribution types

Furthermore, not a single paper reported practicing CVD. In cybersecurity, releasing a "zero-day" exploit without vendor notification is often considered unethical; in NLPSec, it appears to be the standard. Additionally, only 37% of papers acknowledged the potential for malicious misuse of their methodologies.

The study also identified significant biases in the datasets and models targeted.

  • The "English Problem": 97% of the surveyed works focused exclusively on English. Only one paper addressed a low-resource language (Tibetan). This neglect creates a "weakest link" problem where non-English models become easy backdoors for bypassing safety filters.
  • Open Source Gaps: 36% of the research was "functionally closed-source," describing attacks without providing code. This asymmetry hinders "white hat" defenders who must re-implement attacks while malicious actors may already be iterating on the published theory.

Conclusion and Future Directions

To mature as a discipline, NLPSec must adopt more rigorous ethical standards. The authors recommend treating harm minimization as a core principle from project inception and expanding security research to include low-resource languages to prevent geographic safety gaps. Finally, the community needs standardized disclosure protocols, perhaps through AI-specific "Computer Emergency Response Teams" (CERTs).

By bridging the gap between NLP transparency and cybersecurity accountability, the field can move toward a safer, more equitable future.

Summary updated

BibTeX
@article{lent2025nlp,
  author = {Heather C. Lent and Erick Galinkin and Yiyi Chen and Jens Myrup Pedersen and Leon Derczynski and Johannes Bjerva},
  title = {{NLP Security and Ethics, in the Wild}},
  journal = {Trans. Assoc. Comput. Linguistics},
  year = {2025},
  doi = {10.1162/tacl\_a\_00762},
}
Figure