TL;DR
Standard RGB cameras often fail on construction sites due to extreme lighting and motion blur. This paper demonstrates that event cameras, which respond to changes in brightness rather than capturing full frames, significantly outperform RGB in low-light conditions and high-speed scenarios. By fusing both sensors, researchers achieved a "best of both worlds" detector that remains robust across all site conditions.
What Problem Does This Paper Address?
Construction sites are high-risk environments that don't stop when the sun goes down. Safety monitoring and autonomous machinery rely heavily on computer vision, but standard RGB cameras have a major "Achilles' heel": a limited dynamic range (roughly 60 dB).
In the real world, this leads to two fatal failure modes:
- Under-exposure: In low light, cameras increase gain and exposure time, resulting in noisy images and heavy motion blur.
- Over-exposure: High-contrast scenes (like a tunnel exit or floodlights) saturate the sensor, "blinding" the detector.
Event cameras offer a solution with over 120 dB of dynamic range and microsecond latency. However, until now, their specific benefits for construction-relevant object detection (like spotting workers and heavy plant) hadn't been rigorously quantified against a fair RGB baseline on real-world data.
Methods
The researchers compared three state-of-the-art event-based detectors—SMamba (state-space model), SATE (multi-branch temporal encoder), and EvRT-DETR (a transformer-based adapter)—against a YOLOv8x RGB baseline.
Key methodological highlights include:
- Training-Free Late Fusion: Instead of complex "learned" fusion, they simply took the union of detections from both sensors and applied Non-Maximum Suppression (NMS).
- Interpretability Probes: They used "activation patching" to see which internal feature scales (e.g., 1/16 vs. 1/8 resolution) were most damaged by losing event data.
- Controlled Darkening: They mathematically "dimmed" well-lit scenes to observe exactly where RGB collapses compared to the gradual degradation of event sensors.
- Zero-Shot Simulation: They tested the models on synthetic construction sites rendered in Blender to see how well models trained on driving data transfer to the specific geometries of a job site.
Datasets
- PEOD (Pixel-aligned Event-RGB Object Detection): The primary benchmark, featuring 72,000+ frames of synchronized RGB and event data from urban driving, including tunnels and night scenes.
- Simulated Construction Sites: Three custom Blender-rendered environments (Tunnel, Sandy Yard, Rebar Yard) used to test "zero-shot" performance in scenarios like dim floodlighting and dense rebar textures.
Key Insights and Ideas
- Motion is the "Light" for Events: For event cameras, brightness is secondary. What matters is motion. While RGB performance plummets as objects move faster (due to blur), event detectors actually maintain or improve their lead because faster motion generates a stronger, clearer signal.
- The Signal-to-Noise Breaking Point: Event detectors don't fail just because it's dark; they fail when the "signal" (events from moving objects) is drowned out by "noise" (background sensor activity).
- Spatial Redundancy: Event data is incredibly sparse. The team found they could prune 40% of the input tiles while retaining 95% of the detection accuracy. However, they noted a "small object tax": pruning disproportionately hurts the recall of distant workers.
- Scale Specialization: Small objects rely almost entirely on the finest feature scales (1/8 resolution), while larger objects are more resilient and can be detected using coarser (1/16) features even when data is lost.
Key Results
- Low-Light Superiority: In low-light frames (70% of the test set), the best event detector (EvRT-DETR) achieved 27.2 AP, significantly beating the trained RGB baseline's 21.3 AP.
- Fusion Wins: A simple late fusion of both sensors was the top performer in every lighting condition, reaching 33.3 AP overall.
- High-Speed Robustness: For objects moving faster than 20 pixels per frame, RGB only found 11–19% of objects, while EvRT-DETR caught 39–60%.
- Zero-Shot Success: In simulated "dim night" scenarios, the RGB detector's accuracy dropped to near zero, while the event detector retained most of its performance.
Limitations and Future Work
The authors acknowledge that while event cameras excel at motion, they struggle with static objects if the camera itself isn't moving. Furthermore, while the "person" class in the PEOD dataset is a good proxy for workers, it doesn't capture the specific poses or safety gear (PPE) unique to construction.
Future directions include: * Developing native construction datasets that include specific "plant" classes (excavators, loaders). * Exploring more sophisticated feature-level fusion to better handle the transition between day and night. * Investigating why small objects are so fragile under token pruning to enable more efficient edge deployment on battery-powered site cameras.
Summary updated