Low-Light Image Enhancement Framework for Improved Object Detection in Fisheye Lens Datasets

Dai Quoc Tran, Armstrong Aboah, Yuntae Jeon, Maged Shoman, Minsoo Park, Seunghee ParkWorkshops @ CVPR 2024 · DOI · arXiv

Abstract

This paper presents a robust multi-stage framework designed to enhance object detection performance in low-light fisheye lens traffic imagery. By integrating transformer-based image restoration, night-to-day scene conversion, and an ensemble of state-of-the-art detectors, the proposed system addresses the dual challenges of extreme geometric distortion and poor visibility. The framework achieved 5th place in the 2024 AI City Challenge Track 4 with an F1 score of 0.5965, demonstrating significant improvements over baseline methods.

Introduction

Urban traffic monitoring is increasingly adopting fisheye lenses due to their ability to provide 360-degree coverage with a single camera, thereby reducing infrastructure costs. However, these "omnidirectional" cameras introduce significant technical hurdles: extreme radial distortion, peripheral blurriness, and severely degraded visibility during nighttime. Standard object detection models, typically optimized for rectilinear, well-lit imagery, often fail to generalize to these warped, low-signal environments. This work specifically targets the detection of small, distorted objects—such as cars, pedestrians, and bicycles—in low-illumination fisheye datasets where spatial features are heavily compromised.

Proposed Framework

The authors propose a comprehensive three-stage pipeline to bridge the gap between low-quality raw captures and high-accuracy detection requirements.

The proposed three-stage framework: (1) Data Pre-processing with NAFNet and GSAD, (2) Detection Models for Training, and (3) Robust Post-processing Strategy.

Stage 1: Effective Data Pre-processing

The first stage focuses on normalizing the input data. Raw fisheye images are first processed using NAFNet (Non-linear Activation Free Network), a transformer-based architecture, to perform denoising and deblurring. Following restoration, images are categorized into "night" or "other" based on their mean RGB values. Nighttime images undergo a "Night-to-Day" conversion using a Global Structure-Aware Diffusion (GSAD) model. This step effectively "relights" the scene, reducing the domain variance that downstream detection models must handle.

Stage 2: Detection Ensemble

The framework leverages an ensemble of three high-performing detection architectures: YOLOv8x, YOLOv9e, and Co-DETR (Collaborative Detection Transformer). These models are trained on the enhanced, "day-like" dataset, allowing them to learn features from a more consistent and visible domain.

Stage 3: Robust Post-processing

To further refine results, test images are upscaled by 4x using a Dual Aggregation Transformer (DAT) before inference. This super-resolution step helps recover fine details of small objects, particularly at the distorted edges of the fisheye frame. Finally, predictions from the ensemble are merged using Weighted Box Fusion (WBF) to resolve overlapping bounding boxes and improve confidence.

Experimental Setup and Datasets

The framework was evaluated using several key datasets:

  • FishEye8K: A benchmark for fisheye object detection containing 8,000 labeled images.
  • AI City Challenge 2024 (Track 4): The primary competition platform for validation and testing.
  • LOLv2 Synthetic & REDS: Utilized for pre-training the enhancement components (GSAD and NAFNet, respectively).

Results and Analysis

The proposed framework demonstrated both quantitative and qualitative superiority. In the 2024 AI City Challenge Track 4, the system secured 5th place among 52 teams, achieving an F1 score of 0.5965. On the FishEye8K dataset, the final ensemble (Our-2) reached an mAP of 0.489, representing a 21% relative improvement over the 0.4029 baseline.

Qualitative comparison showing the original low-light fisheye image (left) and the enhanced 'day-like' image (right) with corresponding object detections.

Key insights from the results include:

  • Lighting Normalization: Transforming data into a high-illumination domain is more effective than attempting to train light-invariant detectors.
  • Super-Resolution Utility: Applying DAT during inference is crucial for identifying small objects in low-pixel-density regions of the fisheye lens.
  • Ensemble Synergy: Combining one-stage (YOLO) and transformer-based (Co-DETR) detectors via WBF provides complementary strengths that a single model cannot match.

Conclusion and Future Work

While the framework provides a significant leap in detection accuracy for challenging fisheye environments, it introduces substantial computational overhead due to the multiple transformer stages and model ensembles. Future research should focus on developing more efficient, end-to-end architectures that integrate enhancement and detection into a single network to reduce inference latency for real-time traffic monitoring applications. Additionally, addressing potential artifacts introduced during the night-to-day conversion remains a priority for improving photorealism and detection reliability.

Summary updated

BibTeX
@inproceedings{tran2024low,
  author = {Dai Quoc Tran and Armstrong Aboah and Yuntae Jeon and Maged Shoman and Minsoo Park and Seunghee Park},
  title = {{Low-Light Image Enhancement Framework for Improved Object Detection in Fisheye Lens Datasets}},
  booktitle = {Workshops @ CVPR},
  year = {2024},
  doi = {10.1109/cvprw63382.2024.00699},
}
Figure