EO-VLM: VLM-Guided Energy Overload Attacks on Vision Models

Minjae Seo, Myoungsung You, Junhee Lee, Jaehan Kim, Hwanjo Heo, Jintae Oh, Jinwoo KimarXiv 2025 · DOI · arXiv

Introduction

As computer vision models become increasingly integrated into critical infrastructure—ranging from autonomous vehicle navigation to smart city surveillance—their availability and reliability are paramount. Recent research has highlighted a specific vulnerability known as "sponge attacks" or energy-overload attacks. These adversarial techniques aim to maximize a system's computational load, thereby draining batteries or inducing critical latency in real-time decision-making.

Traditional energy-overload attacks typically suffer from two major limitations: they often require white-box access to the target model's internal architecture and are highly specific to a single model, necessitating manual redesign for different targets. To address these gaps, we present EO-VLM, a model-agnostic, black-box framework that leverages the generative power of Vision-Language Models (VLMs) to automate the creation of energy-intensive adversarial images.

Methodology: The VLM as an Adversarial Strategist

The EO-VLM framework utilizes a VLM (such as DALL·E 3) in a dual role: as a vulnerability consultant that identifies architectural weaknesses and as an artist that generates the adversarial content. The process is defined by a four-step iterative loop:

  1. Vulnerability Querying: The adversary queries the VLM to identify factors likely to increase energy consumption in a specific model class (e.g., YOLOv8).
  2. Structured Prompting: Based on the VLM's suggestions—such as increasing anchor box proposals or utilizing steganography—a structured prompt is generated.
  3. Adversarial Generation: The VLM generates an image that incorporates these "computationally heavy" features, often appearing as invisible noise to human observers.
  4. Feedback and Optimization: The energy consumption ($E = W \cdot t$) is measured on the target hardware. If the consumption does not meet the desired threshold, the system iterates with refined prompts.
The iterative EO-VLM framework for generating energy-overload adversarial images.

This approach exploits a significant "safety filter gap" in current commercial VLMs. While these models are designed to block harmful or illegal content, they currently lack mechanisms to prevent the generation of adversarial noise intended for computational denial-of-service attacks.

Experimental Evaluation

The effectiveness of EO-VLM was evaluated against three prominent vision architectures: YOLOv8 (real-time detection), MASKDINO (segmentation), and Detectron2. All tests were conducted using an NVIDIA RTX 4090 GPU.

Key Results

The framework demonstrated a significant ability to degrade performance across all tested models using only a single adversarial image.

Energy Consumption: The most pronounced impact was observed in YOLOv8, which saw a 44.5% increase in power usage when processing steganography-based images. MASKDINO and Detectron2 also exhibited notable increases of 14% and 18.4%, respectively.

Comparison of energy consumption (Watts) across different vision models and attack strategies.

Inference Latency: The slowdown in processing speed was equally dramatic. Detectron2’s latency increased by 50%, jumping from 0.20ms to 0.30ms. MASKDINO experienced a 40.6% increase in processing time. These delays are particularly dangerous for real-time systems like autonomous vehicles, where milliseconds can determine safety outcomes.

Discussion and Future Directions

A key insight from this research is the effectiveness of steganography as an adversarial tool. By hiding data within pixels, the attack forces the target model to expend more cycles attempting to extract features from what appears to be a "clean" background. Furthermore, the study revealed that "computational complexity" for a neural network does not correlate with file size; adversarial images were often smaller than their base counterparts despite requiring significantly more GPU power.

Future work will focus on improving iterative efficiency by incorporating Reinforcement Learning (RL) to optimize prompt selection. More importantly, this research serves as a call to action for VLM providers to develop safety filters capable of identifying and mitigating the generation of energy-overload adversarial content.

Summary updated

Abstract

Vision models are increasingly deployed in critical applications such as autonomous driving and CCTV monitoring, yet they remain susceptible to resource-consuming attacks. In this paper, we introduce a novel energy-overloading attack that leverages vision language model (VLM) prompts to generate adversarial images targeting vision models. These images, though imperceptible to the human eye, significantly increase GPU energy consumption across various vision models, threatening the availability of these systems. Our framework, EO-VLM (Energy Overload via VLM), is model-agnostic, meaning it is not limited by the architecture or type of the target vision model. By exploiting the lack of safety filters in VLMs like DALL-E 3, we create adversarial noise images without requiring prior knowledge or internal structure of the target vision models. Our experiments demonstrate up to a 50% increase in energy consumption, revealing a critical vulnerability in current vision models.
BibTeX
@misc{seo2025eo,
  author = {Minjae Seo and Myoungsung You and Junhee Lee and Jaehan Kim and Hwanjo Heo and Jintae Oh and Jinwoo Kim},
  title = {{EO-VLM: VLM-Guided Energy Overload Attacks on Vision Models}},
  year = {2025},
  eprint = {2504.08205},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2504.08205},
}
Figure