A New Multi-Source Light Detection Benchmark and Semi-Supervised Focal Light Detection

Jae-Yong Baek, Yong-Sang Yoo, Seung Hwan BaeNeurIPS 2024

TL;DR

This paper introduces YouTube Driving Light Detection (YDLD), a massive new benchmark for identifying vehicle lights, traffic signals, and streetlights in diverse driving conditions. To tackle the "tiny object" problem inherent in light detection, the authors propose a Semi-Supervised Focal Light Detection (SS-FLD) framework that leverages a novel loss function and geometric scene priors to significantly outperform state-of-the-art detectors.

What Problem Does This Paper Address?

While autonomous vehicles have become adept at spotting cars and pedestrians, detecting light sources—like a distant brake light or a specific traffic signal—remains a major hurdle.

The challenge is twofold:

  1. Data Scarcity: Most existing light detection (LD) datasets are small and treat all lights as a single "binary" category. In reality, a streetlight and a car's tail light mean very different things for a self-driving system.
  2. The "Tiny Object" Problem: At a distance, light sources are just a few pixels wide. They often look like identical, saturated white blobs, making them extremely difficult to classify and prone to false positives. Standard object detectors (like YOLO or Faster R-CNN) often struggle with these tiny, non-discriminative features.
Urban driving scene showing the tiny object challenge in light detection

Methods

The authors propose a two-pronged solution to improve detection accuracy without requiring an impossible amount of manual labeling:

  • Lightness Focal (LF) Loss: Standard focal loss helps models focus on "hard" samples. The authors modified this into LF Loss, which specifically penalizes false positives more heavily. Since tiny lights are easily confused with background noise, this loss forces the model to be more certain before making a prediction.
  • Lightness Spatial Attention Prior: This is the "aha" moment of the paper. The authors realized that lights follow geometric rules: streetlights are usually high up (in the "sky" region), while car lights are lower (on the "road"). By calculating vanishing points and road geometry, they created a spatial map that tells the model where certain types of lights are likely to be, helping it disambiguate two identical-looking white dots.
  • Semi-Supervised Learning (SS-FLD): To make the most of unlabeled data, they used a "Teacher-Student" framework. A teacher model generates "pseudo-labels" on unlabeled images, which the student model then learns from using the specialized LF loss.

The YouTube Driving Light Detection (YDLD) Dataset

The core of the contribution is the YDLD dataset, designed to push the boundaries of current perception systems:

  • Scale: 3.5k images with 116k high-quality bounding box annotations.
  • Diversity: Images were sourced from YouTube across 19 different countries, covering various night and evening driving scenarios.
  • Classes: Unlike binary datasets, YDLD distinguishes between Carlights, TrafficSignallights, and Streetlights.
  • Comparison: The authors also evaluated their models against standard benchmarks like COCO, KITTI, and BDD100k to prove that pre-training on YDLD improves performance on other driving tasks.
Highway driving scene from the YDLD dataset illustrating diverse lighting conditions

Key Insights and Results

  • Context is King for Tiny Objects: When an object is only 4x4 pixels, its internal texture is useless. The most important information is the global context—where it sits in the scene relative to the horizon and vanishing points.
  • The Geometry of the Road: By using a DBSCAN algorithm to find vanishing points across thousands of frames, the authors proved that light sources cluster in predictable "zones." Integrating this "common sense" into a neural network prevents it from hallucinating a car light in the middle of the sky.
  • Transferability: Interestingly, a model pre-trained on YDLD showed better "transfer learning" results on the BDD100k dataset than models trained on much larger, general datasets like ImageNet. This suggests that specialized light detection is a distinct and necessary feature for robust AI vision.

When tested against 20 different state-of-the-art detectors (including DINO, YOLOX, and various Tiny Object Detectors), the authors' methods showed remarkable improvements:

  • Consistent Boosts: Their SS-FLD method consistently improved the mean Average Precision (mAP) across all detector types.
  • Top Performance: Their best configuration (SS-FLD with RFLA) achieved a 26.0 mAP, a significant jump over the baseline RFLA (21.6 mAP).
  • Tiny Object Mastery: The most dramatic improvements were seen in the "Very Tiny" ($AP_{vt}$) category, where their method nearly doubled the performance of some baseline models by reducing false positives.

Limitations and Future Work

While YDLD is a major step forward, the authors acknowledge several areas for growth:

  • Missing Classes: The current dataset excludes sunlight/glare, indoor lights, and neon signboards. Sunlight, in particular, is difficult to annotate due to its irregular shapes and intense flare.
  • Task Expansion: The current work focuses on bounding boxes. Future iterations could move toward instance segmentation (pixel-perfect outlines) or de-blurring for low-light images.
  • Temporal Logic: The paper primarily looks at static frames. Incorporating video-based "action recognition" (e.g., recognizing a turn signal by its blinking pattern) is a logical next step for the research.

Summary updated

BibTeX
@inproceedings{baek2024new,
  author = {Jae-Yong Baek and Yong-Sang Yoo and Seung Hwan Bae},
  title = {{A New Multi-Source Light Detection Benchmark and Semi-Supervised Focal Light Detection}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2024},
}
Figure