DarkFeat: Noise-Robust Feature Detector and Descriptor for Extremely Low-Light RAW Images

Yuze He, Yubin Hu, Wang Zhao, Jisheng Li, Yong-Jin Liu, Yuxing Han, Jiangtao Wen

Introduction

Robust feature detection and description are fundamental to computer vision tasks such as Simultaneous Localization and Mapping (SLAM) and Structure-from-Motion (SfM). However, these systems frequently fail in extreme low-light environments where the signal-to-noise ratio (SNR) is exceptionally low. Traditional approaches typically employ a "denoise-then-detect" pipeline, utilizing heavy-duty denoising algorithms (e.g., BM3D or deep learning-based SID) before applying standard feature detectors like SIFT or SuperPoint.

These modular pipelines suffer from two primary drawbacks: high computational latency, which precludes real-time application, and the introduction of denoising artifacts that can lead to "ghost" keypoints. To address these challenges, we present DarkFeat, an end-to-end framework designed to detect and describe features directly from low-light RAW images.

Methodology

DarkFeat shifts the paradigm from hiding noise to learning robustness against it. The framework is built upon three core technical pillars:

Direct RAW Processing

Unlike standard RGB-based methods, DarkFeat operates on RAW sensor data. RAW images preserve the original bit depth and sensing information that is often discarded or distorted by a camera's internal Image Signal Processing (ISP) pipeline, providing a richer signal for feature extraction in dark scenes.

Noise-Robustness and Selective Suppression

The architecture, a modified L2-Net, utilizes a dual-branch training strategy with shared weights. One branch processes "normal" (clean) images while the other processes simulated noisy counterparts.

The DarkFeat architecture featuring the Modified L2-Net with dual-branch training and the Selective Suppression Constraint.

A key innovation is the Noise Robustness Map (R), a learned mask that identifies image regions where feature extraction remains reliable despite noise. This is coupled with a Selective Suppression Constraint, which enforces consistency between detection scores of noisy and clean images in "strong" feature regions while actively suppressing scores in noise-prone "weak" regions.

High-Fidelity Synthesis

To overcome the lack of large-scale RAW matching datasets, a synthesis pipeline was developed. It employs an invertible-ISP network to convert high-quality RGB datasets (like GL3D) back to RAW, followed by the injection of realistic, physics-based sensor noise calibrated for professional-grade sensors.

Experimental Results

DarkFeat was evaluated on the Matching In the Dark (MID) benchmark, which includes 108 real-world indoor and outdoor scenes captured under varying ISO and exposure settings.

Performance and Efficiency

DarkFeat demonstrates superior accuracy in camera pose estimation compared to state-of-the-art "denoise-then-detect" baselines. Crucially, by eliminating the separate denoising stage, DarkFeat achieves a 70% reduction in computational cost, running at approximately 13 FPS on standard hardware.

Qualitative comparison of feature matching on the MID benchmark. DarkFeat (bottom) significantly outperforms traditional and deep learning-based pipelines in extreme low light.

Qualitative Robustness

Visual comparisons highlight DarkFeat's ability to ignore noise patterns that typically mislead other detectors. While baselines often match spurious noise artifacts, DarkFeat consistently identifies the underlying structural landmarks of the scene, leading to significantly lower rotation and translation errors.

Conclusion and Future Work

DarkFeat represents the first end-to-end deep learning framework for direct feature extraction from low-light RAW images. By integrating noise handling into the detection process, it offers a high-performance, low-latency solution for autonomous systems operating in the dark. Future research will focus on improving sensor-agnostic generalization and addressing the challenges of combined extreme noise and high-speed motion blur.

Summary updated

Abstract

Low-light visual perception, such as SLAM or SfM at night, has received increasing attention, in which keypoint detection and local feature description play an important role. Both handcraft designs and machine learning methods have been widely studied for local feature detection and description, however, the performance of existing methods degrades in the extreme low-light scenarios in a certain degree, due to the low signal-to-noise ratio in images. To address this challenge, images in RAW format that retain more raw sensing information have been considered in recent works with a denoise-then-detect scheme. However, existing denoising methods are still insufficient for RAW images and heavily time-consuming, which limits the practical applications of such scheme. In this paper, we propose DarkFeat, a deep learning model which directly detects and describes local features from extreme low-light RAW images in an end-to-end manner. A novel noise robustness map and selective suppression constraints are proposed to effectively mitigate the influence of noise and extract more reliable keypoints. Furthermore, a customized pipeline of synthesizing dataset containing low-light RAW image matching pairs is proposed to extend end-to-end training. Experimental results show that DarkFeat achieves state-of-the-art performance on both indoor and outdoor parts of the challenging MID benchmark, outperforms the denoise-then-detect methods and significantly reduces computational costs up to 70%. Code is available at https://github.com/THU-LYJ-Lab/DarkFeat.
BibTeX
@inproceedings{he2023darkfeat,
  author = {Yuze He and Yubin Hu and Wang Zhao and Jisheng Li and Yong-Jin Liu and Yuxing Han and Jiangtao Wen},
  title = {{DarkFeat: Noise-Robust Feature Detector and Descriptor for Extremely Low-Light RAW Images}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2023},
  doi = {10.1609/aaai.v37i1.25161},
}
Figure