Evaluating Large Language Models' Capability to Conduct Cyberattacks On Embedded Devices
Lists
Only you see this
Abstract
As large language models continue to evolve, they have the potential to automate and enhance various aspects of computer security, including red teaming assessments. In this article, we conduct 32 computer security attacks and compare their success rates when performed manually and with assistance from large language models. The security assessments target five connected devices commonly found in modern households (two door locks, one vacuum cleaner, one garage door, and one smart vehicle adapter). We use attacks such as denial-of-service attacks, Man-in-the-Middle, authentication brute force, malware creation, and other common attack types. Each attack was performed twice, once by a human and once by an LLM, and scored for damage, reproducibility, exploitability, affected users, and discoverability based on the DREAD framework for computer security risk assessments. For the LLM-assisted attacks, we also scored the LLM's capacity to perform the attack autonomously. LLMs regularly increased the reproducibility and exploitability of attacks, but no LLM-based attack enhanced the damage inflicted on the device, and the language models often required manual input to complete the attack.
BibTeX
@misc{heiding2025evaluating,
author = {Fred Heiding and Simon Lermen},
title = {{Evaluating Large Language Models' Capability to Conduct Cyberattacks On Embedded Devices}},
year = {2025},
url = {https://openreview.net/forum?id=n2xueVy5ek},
howpublished = {OpenReview},
note = {Submitted to ICLR 2025},
}Loading related content...