LLVIP: A Visible-infrared Paired Dataset for Low-light Vision

Xinyu Jia, Chuang Zhu, Minzhen Li, Wenqi Tang, Wenli Zhou

Abstract

Visual perception in low-light conditions is a critical challenge for applications such as autonomous driving and 24-hour surveillance. Visible-light cameras often lose critical detail in the dark, while infrared sensors provide heat signatures but lack textural richness. This paper introduces LLVIP, a large-scale dataset of over 15,000 strictly aligned visible-infrared image pairs specifically captured in low-light environments. By providing high-resolution, time-synchronized data with pedestrian annotations, LLVIP serves as a robust benchmark for advancing image fusion, object detection, and image-to-image translation in challenging night-time scenarios.

Introduction

Visual tasks like pedestrian detection become notoriously difficult when ambient light is scarce. Visible-light sensors suffer from low contrast and high noise, whereas infrared (thermal) cameras can highlight human heat signatures regardless of lighting. While multi-modal sensor fusion is a logical solution, the field has been hindered by a lack of high-quality data. Existing datasets are often limited in scale, poorly aligned spatially, or captured in well-lit conditions that do not represent the "very dark" scenes where infrared is most valuable.

The LLVIP Dataset

The LLVIP dataset addresses these gaps by providing 15,488 image pairs (30,976 total images) across 26 different urban locations. The images are high-resolution (1920 x 1080 for visible and 1280 x 720 for infrared), offering a significant upgrade over established benchmarks like TNO or KAIST in terms of both scale and clarity.

Overview of the LLVIP dataset showing strictly aligned visible-infrared pairs in various low-light urban scenes.

Methods

To ensure the dataset is research-ready, the authors focused on two critical technical processes:

  1. Strict Registration: Because the two camera sensors have different fields of view, a semi-manual registration process was employed. By selecting corresponding points in both images to calculate a projection transformation, the authors aligned the images perfectly in both space and time.
  2. Cross-Modal Labeling: Labeling pedestrians in pitch-black visible images is nearly impossible for humans. To solve this, the authors labeled pedestrians in the infrared images—where heat signatures make them obvious—and then programmatically mapped those annotations onto the aligned visible images.
Example of a strictly aligned pair from the LLVIP dataset, showing a pedestrian and a motorcycle in a low-light crosswalk.

Key Results and Benchmarking

The authors benchmarked three major tasks on the new dataset:

  • Image Fusion: Testing algorithms like DenseFuse and IFCNN showed that these methods effectively blend thermal signatures with available background textures, providing a more comprehensive view of the scene than either modality alone.
  • Pedestrian Detection: Using YOLOv5, the results were stark. In low light, the infrared-only model achieved an AP50 of 0.965, significantly outperforming the visible-only model (0.908). This confirms that infrared is essential for safety-critical applications at night.
  • Image-to-Image Translation: Results using pix2pixGAN were relatively poor (SSIM of 0.1757), highlighting that current generative models struggle to "hallucinate" meaningful thermal signatures from nearly-black visible pixels, marking a clear area for future algorithmic improvement.

Conclusion

LLVIP provides a foundational resource for multi-modal vision research. By solving the alignment problem and focusing on extreme low-light conditions, it enables more accurate supervised learning for fusion and translation tasks. Future work may expand the dataset to include long-distance targets and more diverse environmental conditions beyond urban surveillance.

Reference: Jia, X., Zhu, C., Li, M., Tang, W., Liu, S., & Zhou, W. (2023). LLVIP: A Visible-infrared Paired Dataset for Low-light Vision. arXiv:2108.10831v4.

Summary updated

Abstract

It is very challenging for various visual tasks such as image fusion, pedestrian detection and image-to-image translation in low light conditions due to the loss of effective target areas. In this case, infrared and visible images can be used together to provide both rich detail information and effective target areas. In this paper, we present LLVIP, a visible-infrared paired dataset for low-light vision. This dataset contains 33672 images, or 16836 pairs, most of which were taken at very dark scenes, and all of the images are strictly aligned in time and space. Pedestrians in the dataset are labeled. We compare the dataset with other visible-infrared datasets and evaluate the performance of some popular visual algorithms including image fusion, pedestrian detection and image-to-image translation on the dataset. The experimental results demonstrate the complementary effect of fusion on image information, and find the deficiency of existing algorithms of the three visual tasks in very low-light conditions. We believe the LLVIP dataset will contribute to the community of computer vision by promoting image fusion, pedestrian detection and image-to-image translation in very low-light applications. The dataset is being released in https://bupt-ai-cz.github.io/LLVIP/.
BibTeX
@inproceedings{jia2021llvip,
  author = {Xinyu Jia and Chuang Zhu and Minzhen Li and Wenqi Tang and Wenli Zhou},
  title = {{LLVIP: A Visible-infrared Paired Dataset for Low-light Vision}},
  booktitle = {ICCVW},
  year = {2021},
  url = {https://openreview.net/forum?id=X5KZOrslBDro},
}
Figure