Cheap night cameras see in the dark by shining their own invisible light, near-infrared (NIR) light at about 850 nm from a small LED ring, and recording it with the same silicon sensor that takes color pictures. Recent methods use that NIR frame as a clean "structure guide" to restore the noisy, color-starved visible frame. They share a hidden assumption: that the NIR frame is equally trustworthy everywhere. It is not. The LED ring makes a bright hot spot in the middle of the view, falls off toward the edges, and falls off again with the square of distance. Near the image border and on far objects, the NIR guide is as noisy as the image it is supposed to fix. The authors of the leading NIR-guided method name uneven NIR illumination as the main problem they left open.
We propose to model the NIR illuminator explicitly. The model estimates, at every pixel, how much NIR light actually reached the scene. It uses that estimate twice. First, it normalizes away the hot spot so the structure guide encodes surface detail instead of lamp shape. Second, and this is the new part, it converts the estimate into a per-pixel confidence. That confidence decides how much the restoration borrows from the NIR frame and how much it relies on the visible frame alone. The claim is that the guide's value is spatially uneven, and that a restorer which knows where the guide is weak will beat both a restorer that trusts it uniformly and one that merely flattens its brightness.
Why this sits at the intersection
The spectral limit is what forces the problem into existence. Silicon stops absorbing light near its bandgap, around 1000 nm. So a single-sensor camera can only borrow photons from the narrow 750–1000 nm band. In practice that means a compact LED illuminator mounted beside the lens, not a broad, uniform ambient source. The narrow band also causes what Wan et al. call "structure vanishing": pigments and materials reflect 850 nm light differently from visible light, so some visible edges are missing in NIR and some NIR edges have no visible counterpart. The non-uniform illumination literature supplies the tools and the warning. Global corrections fail when bright regions dominate the statistics, as Automatic Enhancement of Low Light Level Image on the Basis of a Gaussian Mixture Model shows for histograms. And A Weighted Variational Model for Simultaneous Reflectance and Illumination Estimation shows that unweighted Retinex models, which split an image into surface reflectance times illumination, either oversmooth bright areas or amplify noise in dark ones. Both points motivate weighting by local illumination rather than treating the frame uniformly. The difference here is that the illumination being modeled is the auxiliary sensing light, not the ambient scene light.
Relation to prior work
The closest work is Purifying Low-Light Images via Near-Infrared Enlightened Image. It splits the scene into a color code from the visible frame, a structure code from the NIR frame, and an artifact code that is discarded. It adds a self-compensation loss for structure vanishing, and it states that uneven NIR illumination is unresolved. We keep its disentanglement backbone and change two things. The structure code is computed from an illumination-normalized NIR frame, and its injection is gated by a per-pixel confidence map. Wan et al. treat the NIR frame as uniformly clean; we do not.
Enhancing Low-Light Images Using Infrared Encoded Images takes a different route: it removes the camera's IR-cut filter and restores mixed visible-plus-NIR captures with a normalizing flow, a generative model that learns an invertible map to a simple distribution. Its color loss is a global histogram distance. That work motivates our design by showing that wideband photons help. It also constrains it, because its global loss is exactly the kind of whole-image statistic that uneven lighting breaks.
On the illumination side, Fast Enhancement for Non-Uniform Illumination Images using Light-weight CNNs estimates a smooth illumination map on a coarse bilateral grid with a network of a few hundred parameters. Task-Decoupled Bezier Surface Constraint for Uneven Low-Light Image Enhancement constrains uneven illumination with smooth parametric surfaces. We borrow both ideas, a coarse smooth field and a low-dimensional surface, to represent the NIR irradiance field. We apply them to the auxiliary channel, not the output image.
Rethinking No-reference Image Exposure Assessment from Holism to Pixel argues that exposure error is a per-pixel quantity, not a global score. Its P-IEANet model reports a 40% error reduction over holistic baselines. We take that per-pixel stance and use P-IEANet as an auxiliary, no-reference probe of residual exposure error in our outputs.
Finally, Fabrication of a Chalcogenide Glass Microlens Array for Infrared Laser Beam Homogenization flattens a non-uniform infrared laser beam optically, reaching about 62% transmittance at 1064 nm. It represents the hardware alternative, discussed below.
Core Claim
Write the NIR frame as $N(x) = \rho(x)\,E(x) + n(x)$. Here $\rho(x)$ is the surface's NIR reflectance at pixel $x$, $E(x)$ is the NIR irradiance reaching that surface, and $n(x)$ is sensor noise whose variance grows roughly linearly with $\rho E$ under photon shot noise. The ratio $N/E$ recovers reflectance structure, but its noise standard deviation scales as $1/\sqrt{\rho E}$, which comes from Poisson statistics after division. So normalizing alone makes the dim periphery look equally bright while leaving it much noisier.
We therefore define a confidence
$$w(x) = \frac{\hat E(x)}{\hat E(x) + \tau(x)},$$
where $\hat E$ is the estimated irradiance field and $\tau(x)$ is a learned local noise level derived from the visible frame's brightness. The confidence $w$ multiplies the NIR structure features before they enter the fusion decoder. Where $w$ is low, the decoder must rely on the visible branch and its learned priors.
We claim that confidence gating is what closes the gap in the dim regions of the frame. Normalization on its own does not.
Research Program
The irradiance field $\hat E$ is built from two parts. The first is a device-level angular profile of the illuminator. It can be calibrated once from a single flat-wall capture, or, when no calibration is available, learned as a shared low-order parametric surface per camera model. The second is a smooth per-image residual that captures distance falloff and occlusion. We predict this residual on a coarse grid from the NIR frame and the visible frame together, in the manner of Lv et al.'s illumination branch, and constrain it to be smooth with a Bezier-surface penalty.
Wan et al.'s self-compensation loss for structure vanishing is reweighted by $w$. The network is then not pushed to match NIR edges in places where those edges are mostly noise.
Training uses the 1,200 NIR-enhanced training pairs of Wan et al. We add a synthetic augmentation. We identify the most uniformly lit NIR frames, re-illuminate them with randomized radial hot-spot profiles and distance falloff, and inject matching shot noise. This gives exact ground truth for $E$, so the field estimator can be checked directly.
We will also capture a small real test set with the same consumer camera models: Wyze Cam V2 and Eufy. Each scene gets a flat-wall calibration frame, a long-exposure visible reference, and a rough depth map. This set exists so that gains can be attributed to illuminator modeling on real hot spots, not to synthetic artifacts.
Baselines
The naive reference point is visible-only enhancement: a zero-reference curve-estimation method such as ZeroDCE, and Lv et al.'s lightweight non-uniform enhancer, both run on the visible frame alone. These show how much the NIR guide is worth at all. Wan et al. list ZeroDCE, EnlightenGAN and KinD++ as baselines, but no reported result was found on Lacuna for their absolute scores on the NIR-enhanced evaluation set, so we will rerun them.
The strongest existing method is Wan et al.'s purification network, which reports 19.63 dB PSNR and 0.867 SSIM on its 100-triplet evaluation set (Purifying Low-Light Images via Near-Infrared Enlightened Image). We will retrain it with our augmentation, the same number of optimization steps and the same hyperparameter search budget as our model. This separates the effect of the augmentation from the effect of the architecture. We will report both the published number and our reproduction.
The key ablation removes the confidence gate but keeps normalization. The model sees $N/\hat E$ and nothing else changes. If this matches the full model, the core claim is false.
A second ablation replaces the estimated $\hat E$ with the ground-truth field on synthetic data. This bounds how much is lost to estimation error.
Beating the strongest baseline means a statistically reliable PSNR and SSIM gain over the reproduced Wan et al. model on the full evaluation set. It also means a larger gain in the outer rings of the image, where estimated NIR irradiance falls below half its peak, than in the central hot spot. A uniform gain would suggest a generic capacity effect, not the proposed mechanism.
Why not just…?
Why not just divide by a flat-field calibration frame? This is the most serious alternative, and it may capture much of the gain on the dataset's close-range scenes. Division removes the lamp's angular profile. It cannot remove inverse-square falloff, which depends on scene depth, and it amplifies periphery noise instead of discounting it. We include "Wan et al. plus flat-field division" as a baseline. If it matches our model on the depth-varied real test set, the reliability gate is not earning its place and we will say so.
Why not just run an off-the-shelf non-uniform illumination enhancer on the NIR frame first? Methods like Lv et al.'s network or SAIGFormer: A Spatially-Adaptive Illumination-Guided Network for Low-Light Image Enhancement brighten dim regions. They were trained on visible images and do not know the illuminator's geometry. Like flat-fielding, they produce a brighter but equally unreliable periphery. We test this directly as the "enhance-then-fuse" baseline, run under the same training budget.
Why not just homogenize the illuminator optically? The chalcogenide microlens arrays of Zhang et al. were designed for mid-infrared transmission, which is unnecessary at 850 nm, where ordinary glass or plastic diffusers work. Optical flattening spreads a fixed photon budget more evenly: the center gets darker and the edges brighter. It still leaves distance falloff, and existing cameras cannot be retrofitted. This alternative is plausible for new hardware, so on the real test set we will compare it against our method by capturing a subset of scenes with a diffuser fitted.
Proposed Experiments
The decisive check is ring-stratified evaluation. We bin pixels of the 100 NIR-enhanced evaluation triplets and of our real test set by estimated NIR irradiance, then plot PSNR per bin for four systems: visible-only enhancement, Wan et al., normalization-only, and the full model. The hypothesis predicts that all NIR-guided systems converge in the bright center and separate in the dim bins, with the full model degrading gracefully toward the visible-only curve instead of falling below it.
The second check asks whether the field estimator recovers the true irradiance on synthetic re-illuminated data, measured by relative error against the ground-truth field. It also asks whether estimated fields from the same physical camera are consistent across scenes, which they should be if the method learned the lamp rather than the scene.
Third, we report P-IEANet's no-reference residual exposure maps on the outputs, to confirm that the center-to-edge brightness gradient has been removed without new over-exposure in the hot spot.
The claim fails if the normalization-only ablation equals the full model in the dim bins, or if flat-field division equals it on the depth-varied scenes.
References
- Automatic Enhancement of Low Light Level Image on the Basis of a Gaussian Mixture Model
- A Weighted Variational Model for Simultaneous Reflectance and Illumination Estimation
- Purifying Low-Light Images via Near-Infrared Enlightened Image
- Enhancing Low-Light Images Using Infrared-Encoded Images
- Fast Enhancement for Non-Uniform Illumination Images using Light-weight CNNs
- Task-Decoupled Bezier Surface Constraint for Uneven Low-Light Image Enhancement
- Rethinking No-reference Image Exposure Assessment from Holism to Pixel: Models, Datasets and Benchmarks
- Fabrication of a Chalcogenide Glass Microlens Array for Infrared Laser Beam Homogenization
- SAIGFormer: A Spatially-Adaptive Illumination-Guided Network for Low-Light Image Enhancement