How Far Can Frozen Vision-Language Models Go on Event Data? A Training-Free Study on EventBench

Md Kamrujjaman Mobin, Md Salman Shamil, Md Adnan ArefeenarXiv 2026 · arXiv

TL;DR

This paper demonstrates that off-the-shelf, frozen Vision-Language Models (VLMs) can understand event-camera data without any specialized training. By simply rendering event streams as red/blue image grids and using clever prompting, the authors achieved a competitive 62.28% accuracy on the EventBench challenge, proving that general-purpose vision models possess surprising zero-shot reasoning capabilities for non-standard visual modalities.

What Problem Does This Paper Address?

Event cameras are a different breed of sensor. Unlike traditional cameras that capture full frames at fixed intervals, event cameras asynchronously report per-pixel brightness changes (events). This results in a sparse, high-speed stream of data that is fundamentally incompatible with the dense, frame-based architectures of modern multimodal large language models (MLLMs).

Most researchers bridge this gap by training custom event encoders or fine-tuning models on massive event datasets. However, this is computationally expensive and requires significant labeled data. The authors of this paper ask a provocative question: How far can we get with a completely training-free approach? Can we simply "show" event data to a standard, RGB-pretrained VLM in a way it understands?

Methods

The authors developed a simple, training-free pipeline to adapt event data for a frozen Qwen3.5 9B model.

  1. Event-to-Image Rendering: Raw events are grouped into "bins." For each bin, "ON" events (brightness increases) are colored red and "OFF" events (decreases) are colored blue against a white background.
  2. Bin Selection & Grid Stitching: Instead of feeding a video, they select a representative subset of bins (typically 6–8) and stitch them into a single grid image. This allows the model to see the temporal progression of the event stream in a single inference call.
  3. Prompting: They use short, category-specific system prompts that explain the color encoding and the temporal order of the grid cells.
  4. Action Recognition Exception: For the action recognition task, they used the organizers' pre-trained EventGPT-Plus-2B model as-is, feeding it the full stack of rendered frames.
Figure 1. Overview. (a) For four categories, event bins are rendered as images, a category-specific subset is selected and stitched into one grid (shown: the exact model input for a detailed-understanding clip of a woman adjusting her hair, used only in this figure), and a frozen vision-language model answers from it; action recognition passes every bin to the organizers’ EventGPT-Plus-2B. (b) Accuracy on the held-out test set; chance is 25% for the four multiple-choice tasks. The overall accuracy pools correct answers across all 2,084 questions. Without any training data, the system reaches 62.28% overall accuracy, rank 9 of 27.

Datasets

The system was evaluated on the EventBench benchmark (General Understanding track) as part of the EBMV @ ECCV 2026 challenge. The benchmark consists of 2,084 test questions across five categories:

  • Gesture Recognition: Identifying hand movements (e.g., waving, clapping).
  • Detailed Understanding: Answering specific questions about scene content.
  • Action Recognition: Identifying human actions (e.g., pushing, walking).
  • Causal Reasoning: Reasoning about cause-and-effect in event streams.
  • Event OCR: Recognizing text from event data.

These tasks involve a mix of simulated data (V2E) and real-world event camera recordings.

Key Insights and Ideas

  • Zero-Shot Modality Transfer: The most striking finding is that a model trained on natural RGB images can reason over synthetic red/blue edge maps. This suggests that VLMs learn high-level spatial and temporal concepts that are somewhat invariant to the specific input modality.
  • Temporal Reasoning via Spatial Layout: By arranging time-ordered bins in a grid, the authors effectively converted a temporal problem into a spatial one that the VLM's attention mechanism could handle.
  • The "Noise Floor" Problem: A simple bounding-box crop based on min/max event coordinates almost always failed to tighten around the subject. A few stray noise events at the sensor edges were enough to force the crop to cover the entire canvas, leading to significant downsampling when the full frame was squeezed into a small grid cell.
  • Positional Bias: The authors found that for "Detailed Understanding," the model's predictions were heavily skewed toward the first two answer options (A and B), suggesting a need for better answer-order calibration in zero-shot settings.
Examples of the five task categories and the model's predictions.

Key Results

The training-free system achieved an overall accuracy of 62.28%, placing it 9th out of 27 teams in the challenge. This is particularly impressive given that many higher-ranking teams used extensive fine-tuning.

Performance varied significantly by task:

  • Action Recognition: 80.27% (highest, using a specialized but frozen event model).
  • Causal Reasoning: 77.22% (demonstrating strong reasoning capabilities).
  • Detailed Understanding: 66.46%.
  • Gesture Recognition: 54.70% (limited by the low temporal resolution of the 8-bin grid).
  • Event OCR: 34.57% (lowest, primarily due to severe downsampling in the grid cells).

Limitations and Future Work

The primary bottleneck for tasks like OCR and gesture recognition was resolution and temporal sampling. Squeezing a high-resolution event stream into a small grid cell causes fine details (like text strokes) to vanish.

Visualization of the grid input, showing how OCR text becomes heavily downsampled.

Future work could explore:

  • Adaptive Rendering: Adjusting the rendering based on event density to avoid saturation or empty frames.
  • Better Cropping: Using more robust, percentile-based cropping to ignore noise and focus on the active region.
  • Tiling: Instead of a single grid, using a tiling approach to preserve higher resolution for OCR tasks.
  • Calibration: Implementing techniques to mitigate the model's positional bias in multiple-choice questions.

Summary updated

Abstract

We present a training-free system for the EBMV @ ECCV 2026 challenge "Event Stream Understanding with EventBench" (General Understanding track), which evaluates vision-language models on five event-camera understanding tasks. We use none of the challenge's training data: our goal is to measure how far off-the-shelf, RGB-pretrained models can go on event data through representation and prompting alone. For four of the five categories (gesture recognition, detailed understanding, causal reasoning, and event OCR) we render event bins as red/ON, blue/OFF frames, select a category-specific subset of bins, stitch them into a single grid image, and query the same frozen Qwen3.5 9B vision-language model with a category-specific prompt for all four tasks. The fifth category, action recognition, feeds every bin directly into the organizers' EventGPT-Plus-2B. The system reaches 62.28% overall accuracy across 2,084 test questions (rank 9/27), led by action recognition and causal reasoning (80.27%, 77.22%) and limited most by gesture recognition and event OCR (54.70%, 34.57%). We visualize test samples from every category together with the exact images the model receives, score six alternative representations against the evaluation server, and analyze the full test set post hoc. That analysis shows that our bounding-box crop almost never tightened around content, so event OCR text reached the model at roughly one fifth of its horizontal resolution, and that detailed-understanding predictions are skewed toward the first two answer options.
BibTeX
@misc{mobin2026how,
  author = {Md Kamrujjaman Mobin and Md Salman Shamil and Md Adnan Arefeen},
  title = {{How Far Can Frozen Vision-Language Models Go on Event Data? A Training-Free Study on EventBench}},
  year = {2026},
  eprint = {2611.15756},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.15756},
}
Figure