Abstract
This paper considers matching images of low-light scenes, aiming to widen the frontier of Structure-from-Motion (SfM) and visual SLAM applications. Recent image sensors can record the brightness of scenes with more than eight-bit precision, available in their RAW-format image. We are interested in making full use of such high-precision information to match extremely low-light scene images that conventional methods cannot handle. For extreme low-light scenes, even if some of their brightness information exists in the RAW format images’ low bits, the standard raw image processing on cameras fails to utilize them properly. As was recently shown by Chen et al., CNNs can learn to produce images with a natural appearance from such RAW-format images. To consider if and how well we can utilize such information stored in RAW-format images for image matching, we have created a new dataset named MID (matching in the dark). Using it, we experimentally evaluated combinations of eight image-enhancing methods and eleven image matching methods consisting of classical/neural local descriptors and classical/neural initial point-matching methods. The results show the advantage of using the RAW-format images and the strengths and weaknesses of the above component methods. They also imply there is room for further research.
Introduction
Most computer vision systems, like those used in autonomous vehicles or drones, rely on Structure-from-Motion (SfM) and Visual SLAM (Simultaneous Localization and Mapping). These techniques work by finding matching "keypoints" between different images to calculate camera movement and 3D structure. However, these systems usually break down at night. While modern 14-bit image sensors record a vast amount of light information, the standard Raw Image Processing (RIP) pipeline inside cameras typically discards the "lower bits" of data to produce a clean, 8-bit RGB image. In extreme low light, this process often results in images that are almost entirely black. The authors argue that the information needed for matching is still there—hidden in the RAW signal—but our current software pipelines are throwing it away.
The MID Dataset
The core contribution of this work is the MID (Matching In the Dark) dataset. It consists of 10,584 images covering 108 scenes (54 indoor, 54 outdoor). For every scene, the researchers captured stereo pairs at 48 different exposure settings, varying ISO and shutter speed from mildly underexposed to pitch black. To provide a gold standard for evaluation, a high-quality, long-exposure image was captured for every position to calculate the "true" camera pose. Unlike existing datasets, MID contains scenes that are significantly darker, where standard processing yields zero usable information.
Methodology
The researchers conducted an extensive evaluation of 88 combinations of image enhancers and matching algorithms. The pipeline consisted of four stages:
- Image Enhancement: Standard camera processing (RIP) was compared against "Direct" methods working on RAW data, including classical techniques (Histogram Equalization, BM3D denoising) and deep-learning models (SID, MIRNet).
- Keypoint Detection & Description: Eleven methods were tested, including classical descriptors like RootSIFT and ORB, and modern neural networks like SuperPoint (SP), R2D2, and GIFT.
- Initial Matching: Standard Nearest Neighbor (NN) search and the advanced SuperGlue (SG) graph neural network were evaluated.
- Outlier Removal: RANSAC and neural-guided NG-RANSAC were used to filter matches and estimate camera poses.
Results and Discussion
The primary finding is that using RAW data directly is vastly superior to using images processed by the camera's standard RIP. The "lower bits" contain structural details necessary for matching that RIP deletes. The combination of SuperPoint and SuperGlue consistently outperformed other methods.
Interestingly, visual beauty does not equate to matching utility. A "natural-looking" brightened image (e.g., from the SID model) isn't always best for matching; classical denoising (BM3D) applied to RAW data performed surprisingly well. Furthermore, manual tuning of the RAW bit-range outperformed automated enhancers, suggesting that current AI enhancers are optimized for human aesthetics rather than geometric accuracy.
The success rate was measured using $N_\tau$, the percentage of exposure settings where a method estimated the camera pose within a 5-degree error threshold. In indoor scenes, RIP-based methods succeeded only 22% of the time, while RAW-based enhancement (Direct-BM3D) reached nearly 60%. In outdoor scenes, the top combo (SID + SuperPoint + SuperGlue) achieved 57.5%, compared to 30.2% for the best RIP-based method.
Conclusion
The MID dataset demonstrates that RAW data is the path forward for low-light computer vision. However, the gap between manual tuning and automated enhancers indicates a need for new image enhancement models specifically designed to preserve geometric features for SLAM and SfM, rather than just aesthetic quality. Future work may involve end-to-end networks that process RAW data directly for camera pose estimation.
Summary updated