Event-based object detection under adverse lighting and motion: lessons for construction sites

Zhuoran Feng, Xiaoshan ZhouarXiv 2026 · arXiv

TL;DR

Standard RGB cameras often fail on construction sites due to extreme lighting and motion blur. This paper demonstrates that event cameras, which respond to changes in brightness rather than capturing full frames, significantly outperform RGB in low-light conditions and high-speed scenarios. By fusing both sensors, researchers achieved a "best of both worlds" detector that remains robust across all site conditions.

What Problem Does This Paper Address?

Construction sites are high-risk environments that don't stop when the sun goes down. Safety monitoring and autonomous machinery rely heavily on computer vision, but standard RGB cameras have a major "Achilles' heel": a limited dynamic range (roughly 60 dB).

In the real world, this leads to two fatal failure modes:

  1. Under-exposure: In low light, cameras increase gain and exposure time, resulting in noisy images and heavy motion blur.
  2. Over-exposure: High-contrast scenes (like a tunnel exit or floodlights) saturate the sensor, "blinding" the detector.

Event cameras offer a solution with over 120 dB of dynamic range and microsecond latency. However, until now, their specific benefits for construction-relevant object detection (like spotting workers and heavy plant) hadn't been rigorously quantified against a fair RGB baseline on real-world data.

Methods

The researchers compared three state-of-the-art event-based detectors—SMamba (state-space model), SATE (multi-branch temporal encoder), and EvRT-DETR (a transformer-based adapter)—against a YOLOv8x RGB baseline.

Key methodological highlights include:

  • Training-Free Late Fusion: Instead of complex "learned" fusion, they simply took the union of detections from both sensors and applied Non-Maximum Suppression (NMS).
  • Interpretability Probes: They used "activation patching" to see which internal feature scales (e.g., 1/16 vs. 1/8 resolution) were most damaged by losing event data.
  • Controlled Darkening: They mathematically "dimmed" well-lit scenes to observe exactly where RGB collapses compared to the gradual degradation of event sensors.
  • Zero-Shot Simulation: They tested the models on synthetic construction sites rendered in Blender to see how well models trained on driving data transfer to the specific geometries of a job site.

Datasets

  • PEOD (Pixel-aligned Event-RGB Object Detection): The primary benchmark, featuring 72,000+ frames of synchronized RGB and event data from urban driving, including tunnels and night scenes.
  • Simulated Construction Sites: Three custom Blender-rendered environments (Tunnel, Sandy Yard, Rebar Yard) used to test "zero-shot" performance in scenarios like dim floodlighting and dense rebar textures.

Key Insights and Ideas

  • Motion is the "Light" for Events: For event cameras, brightness is secondary. What matters is motion. While RGB performance plummets as objects move faster (due to blur), event detectors actually maintain or improve their lead because faster motion generates a stronger, clearer signal.
  • The Signal-to-Noise Breaking Point: Event detectors don't fail just because it's dark; they fail when the "signal" (events from moving objects) is drowned out by "noise" (background sensor activity).
  • Spatial Redundancy: Event data is incredibly sparse. The team found they could prune 40% of the input tiles while retaining 95% of the detection accuracy. However, they noted a "small object tax": pruning disproportionately hurts the recall of distant workers.
  • Scale Specialization: Small objects rely almost entirely on the finest feature scales (1/8 resolution), while larger objects are more resilient and can be detected using coarser (1/16) features even when data is lost.

Key Results

  • Low-Light Superiority: In low-light frames (70% of the test set), the best event detector (EvRT-DETR) achieved 27.2 AP, significantly beating the trained RGB baseline's 21.3 AP.
  • Fusion Wins: A simple late fusion of both sensors was the top performer in every lighting condition, reaching 33.3 AP overall.
  • High-Speed Robustness: For objects moving faster than 20 pixels per frame, RGB only found 11–19% of objects, while EvRT-DETR caught 39–60%.
  • Zero-Shot Success: In simulated "dim night" scenarios, the RGB detector's accuracy dropped to near zero, while the event detector retained most of its performance.

Limitations and Future Work

The authors acknowledge that while event cameras excel at motion, they struggle with static objects if the camera itself isn't moving. Furthermore, while the "person" class in the PEOD dataset is a good proxy for workers, it doesn't capture the specific poses or safety gear (PPE) unique to construction.

Future directions include: * Developing native construction datasets that include specific "plant" classes (excavators, loaders). * Exploring more sophisticated feature-level fusion to better handle the transition between day and night. * Investigating why small objects are so fragile under token pruning to enable more efficient edge deployment on battery-powered site cameras.

Summary updated

Abstract

Vision-based monitoring on construction sites fails at night, at dawn and dusk, in tunnels and under floodlights, when conventional RGB cameras under- or over-expose. Event cameras record per-pixel brightness changes with microsecond latency and over 120 dB of dynamic range, but their benefit for site-relevant object detection has not been quantified on real data. We compare three recent event-based detectors (SMamba, SATE and EvRT-DETR) with an RGB detector trained on the same data, using the pixel-aligned PEOD dataset. On low-light frames (70% of the test set) the best event detector reaches 27.2 AP against 21.3 AP for RGB, and a training-free late fusion of both sensors is the best option in every lighting condition (33.3 AP). An object-level analysis of 53,091 objects shows that brightness has little effect on event detectors, whereas fast motion matters: above 20 pixels per frame RGB finds 11-19% of medium-size objects and EvRT-DETR 39-60%. Activation patching locates the damage caused by losing events at the 1/16-resolution features, with small objects depending on the finest scale, and shows that event detectors fail when few signal events meet added noise. A model-agnostic pruning study keeps about 95% of accuracy with 60% of the input but halves small-object recall. On rendered construction sites, used zero-shot, dimming the floodlights drops RGB to near zero while the event detector keeps most of its AP50. We close with camera-placement guidance for construction sites.
BibTeX
@misc{feng2026event,
  author = {Zhuoran Feng and Xiaoshan Zhou},
  title = {{Event-based object detection under adverse lighting and motion: lessons for construction sites}},
  year = {2026},
  eprint = {2611.19746},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.19746},
}
Figure