TL;DR
PiLLVD is a new, publicly available dataset of paired low-light and normal-light videos featuring complex, real-world motion captured with a low-cost Raspberry Pi setup. By providing temporally aligned sequences of dynamic scenes, it offers a rigorous benchmark for evaluating how enhancement algorithms handle temporal stability and motion artifacts—areas where current state-of-the-art models still struggle to generalize.
What Problem Does This Paper Address?
While low-light image enhancement has seen massive progress, video enhancement remains a significant challenge. The primary hurdle isn't just brightening a frame; it’s maintaining temporal consistency. If you enhance each frame independently, the video often suffers from "flickering," color shifts, and inconsistent brightness.
To train and evaluate models that solve these issues, researchers need "paired" data: a low-light video and a perfectly aligned normal-light version of the same scene. However, capturing this is technically difficult. You need two cameras perfectly synchronized in time and space. Consequently, most existing datasets are either static (no motion), synthetic (adding fake noise to bright videos), or private, which limits the development of models capable of handling the messy, unpredictable motion of the real world.
Methods
The authors developed a reproducible, low-cost acquisition system to bridge this data gap.
- Hardware Setup: The system uses three Raspberry Pi 4Bs and two identical Arducam Sony-IMX519 cameras. A beam splitter divides incoming light, sending 30% to the "low-light" camera and 70% to the "normal-light" reference camera.
- Synchronization: One Pi acts as a master server, using the Network Time Protocol (NTP) to sync the clocks of the other two "slave" Pis. This ensures a temporal offset of less than one millisecond between paired frames.
- Post-Capture Alignment: Even with a rigid mount, tiny spatial misalignments occur. The authors developed a pipeline that uses histogram matching to temporarily brighten the low-light frames (preserving their structure) to estimate optical flow. They then warp the normal-light reference frame to match the low-light input, ensuring the ground truth is spatially aligned without distorting the original low-light data.
Datasets
The paper introduces PiLLVD (Pi-based Low-Light Video Dataset).
- Composition: 24 paired sequences (17 indoor, 7 outdoor) totaling 4,531 frames.
- Diversity: Unlike previous datasets that use controlled camera rails or static scenes, PiLLVD features uncontrolled human and object motion, such as walking, gesturing, and moving handheld objects.
- Comparison: Compared to existing benchmarks like DRV or SDSD, PiLLVD is categorized as having "High" motion diversity and "Uncontrolled" motion patterns, making it a more realistic test for real-world applications like surveillance or autonomous systems.
Key Insights and Ideas
- The "Zero-Shot" Reality Check: The authors benchmarked several top-tier models (like SDSDNet and StableLLVE) without any fine-tuning on their new data. They found that models trained on static or synthetic data often fail when faced with real-world dynamic scenes.
- PSNR Isn't Everything: Some models achieved high PSNR (Peak Signal-to-Noise Ratio) but performed poorly on temporal stability metrics. This highlights that a "bright" frame isn't necessarily a "good" video frame if it jitters or loses texture.
- Motion Artifacts: The dataset reveals specific failure modes in current models, such as "ghosting" (where a moving person leaves a trail) or over-smoothing of textures in motion.
Key Results
The quantitative evaluation used standard metrics (PSNR, SSIM, LPIPS) alongside mTWE (masked temporal warping error) to measure stability.
- Indoor Performance: LLVE-STCD generally led in structural similarity (SSIM) and perceptual quality (LPIPS) for indoor scenes.
- Temporal Stability: DP3DF demonstrated the strongest temporal consistency (lowest mTWE) across most sequences, proving more robust against flickering.
- Qualitative Failures: In outdoor tests, SDSDNet significantly increased brightness but introduced noticeable ghosting and color smearing around moving subjects.
Limitations and Future Work
The authors acknowledge several areas for improvement:
- Scale: While diverse, the dataset is relatively small (4,531 frames).
- Scene Balance: There is a higher proportion of indoor scenes compared to outdoor ones.
- Alignment Challenges: In extremely dark conditions, the high sensor noise makes it difficult for even advanced optical flow algorithms to achieve perfect pixel-level alignment.
- Future Directions: The team plans to expand the dataset with faster motion, more extreme low-light conditions, and more outdoor environments. They also aim to develop new, efficient models specifically trained on PiLLVD for real-time edge deployment.
Summary updated