When a phone records video that alternates between a dark and a bright exposure to capture high dynamic range (HDR, scene brightness spanning more than one 8-bit image can hold), moving people and objects keep uncovering background they used to hide. Often that newly revealed background appears only in frames where it is pure white, because the long exposure saturated, or pure black, because the short exposure underexposed it. Current methods handle such pixels in one of two ways. Either they discard them and let a generative model invent the content, or they fit the clipped value directly. The first approach hallucinates colors that can contradict what the camera actually recorded. The second leaves translucent "ghost" remnants of the object that used to stand there.
The proposal starts from a simple observation: a clipped pixel is not missing data. It is an inequality. A saturated pixel says the true brightness was at least a certain value, and a black pixel says it was at most a certain value. We propose to carry these one-sided bounds, which statisticians call censored observations, all the way through a dynamic 4D reconstruction and into the generative step that fills disoccluded regions. The diffusion model, a generator that learns by gradually denoising random noise into samples, may then only propose completions that agree with every bound the camera recorded. The claim is narrow and testable. On revealed surfaces that the camera saw only in clipped form, bound-respecting completion should cut both bound violations and ghosting compared with masking-plus-inpainting and with a saturating tone-mapping model alone.
Where this sits
Two lines of work meet here without covering the case. Mono4DGS-HDR builds 4D Gaussian splats (a scene made of many small colored 3D blobs that move over time) from unposed alternating-exposure monocular video. It enforces consistent radiance across adjacent frames using optical flow, but it has no generative fill for disoccluded regions, and its own authors note that it collapses where depth and flow fail in clipped zones (Mono4DGS-HDR). HDR-NSFF models a continuous space-time radiance field with a learnable camera response function (the curve mapping physical brightness to recorded pixel values), tracks motion with exposure-invariant DINOv2 flow, and adds a diffusion regularizer for saturated or unobserved regions (HDR-NSFF). HDR-NSFF is the closest prior work, and it is the main threat to novelty, as discussed below. It needs COLMAP poses, and nothing in the file indicates that its diffusion prior is constrained by the clipping bounds. The prior is a soft pull toward plausible texture, so it can pull radiance below a saturation floor that the data established.
On the disocclusion side, CogNVS reprojects visible pixels into the novel view, fills the holes with a video diffusion model (CogVideoX), and fine-tunes that model at test time on the input video (Reconstruct, Inpaint, Finetune). FreeOrbit4D completes the unseen back of moving objects with multi-view diffusion and renders depth scaffolds that guide a video generator (FreeOrbit4D). Both supply the method we reuse, a hole-conditioned video inpainter. Both also work entirely in 8-bit standard range and have no notion of physical radiance. Classical HDR deghosting motivates the problem: attention-based rejection in ADNet and dual-branch suppression in DRHDR remove ghosts by throwing away conflicting pixels, and Attention-Guided Progressive Neural Texture Fusion documents exactly how a moving object can be well-exposed in one frame and clipped in another, which breaks pixel correspondence.
Monocular scale ambiguity enters as a constraint, not as the core contribution. Where a disocclusion boundary falls depends on depth. Single-image metric depth models such as Metric3D v2 and ZeroDepth predict each frame independently. In alternating exposures, the dark and bright frames can plausibly receive systematically different scales, and the boundary of a revealed region would then oscillate with the exposure. We handle this with a deliberately simple fix and measure how much it matters. We do not claim it as new.
Core Claim
Let $r$ be the HDR radiance of a static background point. Let $g$ be the camera response curve and $\Delta t_k$ the exposure time of input frame $k$. Let $\epsilon$ be a small margin from the clipping limits of the 0–1 pixel range. Every frame $k$ in which the point is visible but clipped contributes one bound. A saturated observation implies $r \ge g^{-1}(1-\epsilon)/\Delta t_k$, and a black one implies $r \le g^{-1}(\epsilon)/\Delta t_k$. Intersecting all bounds gives a feasible interval $[L, U]$ per point. That interval may be half-open or, when the point appeared saturated in the short exposure and black in the long one, empty, which flags a correspondence error. The reconstruction penalizes radiance outside the interval with a hinge loss, and its weight $\lambda$ is set on held-out sequences:
$$\mathcal{L}_{\text{cens}} = \lambda \sum_{p} \big[\max(0, L_p - r_p) + \max(0, r_p - U_p)\big],$$
Here $p$ indexes background points. Well-exposed observations keep the usual photometric loss.
The new step is how the generative fill is made to respect $[L, U]$. For a disoccluded region in a target view, we render the current HDR estimate at each exposure in the bracket and run the hole-conditioned video inpainter on each render. We convert each filled pixel back to radiance through $g^{-1}$, keeping only values from exposures where the filled pixel lies in the unclipped mid-range, and project the result onto $[L, U]$ before using it as supervision. Completion therefore happens in radiance space, in the exposure that can actually express the answer, and the camera's bounds act as hard limits on the hallucination instead of a soft regularizer competing with it.
Research Program
The system has three parts, each kept simple so the ablations are clean. The first is a static-plus-dynamic Gaussian reconstruction initialized as in Mono4DGS-HDR. It uses delayed densification in the spirit of RobustSplat, so that moving objects do not spawn floating blobs in the background before it stabilizes. The second is a visibility ledger. For each static point it records which input frames saw it and whether each sighting was well-exposed, saturated, or black, which yields the interval $[L, U]$ and a label: never seen, seen only clipped, or seen well-exposed at least once. The third is exposure-bracketed, bound-projected inpainting, which uses CogNVS-style test-time fine-tuning of the inpainter on the masked input video.
For scale, we take Metric3D v2 depth only from whichever frame in each bracket pair has more pixels in the mid-range at a given location. We then fit a single global scale per sequence to the static points tracked across both exposures, using a least-squares alignment of the kind Wofk et al. use with sparse points. The analysis question is whether exposure-dependent scale drift measurably shifts disocclusion masks. If it does not, this component is dropped.
Baselines
The do-nothing reference is Mono4DGS-HDR run as published, with no generative fill. Disoccluded regions get whatever its radiance field produces. This shows how bad ghosting and holes are in the target subset when no generation is attempted. On its own synthetic alternating-exposure benchmark, Mono4DGS-HDR reports HDR PSNR 37.64 against 36.89 for the next best method (Mono4DGS-HDR).
The strongest existing competitor is HDR-NSFF. It already combines a learnable saturating tone curve with a diffusion prior over saturated and unobserved regions, so it is the method most likely to make our contribution redundant. It reports PSNR 32.63 against 29.70 for the next best baseline on HDR-GoPro (HDR-NSFF). To isolate the bound-handling question from the representation, we will reimplement its tone-mapping and diffusion-regularization recipe on top of our Gaussian reconstruction. We will also run the original where poses are available. Both variants receive the same poses, the same base inpainter, and the same number of hyperparameter trials and GPU-hours as our method.
The cascade baseline runs Mono4DGS-HDR, then CogNVS inpainting on a mid-exposure LDR render, then inverse tone mapping through the learned curve. CogNVS reports PSNR 22.63 against 20.93 for TrajectoryCrafter on Kubric-4D (Reconstruct, Inpaint, Finetune). No reported result was found on Lacuna for CogNVS on any HDR or alternating-exposure benchmark.
The key ablations remove one ingredient at a time. The first keeps bracketed inpainting but drops the projection onto $[L, U]$. The second keeps $\mathcal{L}_{\text{cens}}$ but inpaints a single exposure. The third replaces censoring with masking of clipped pixels. Beating the baselines means a statistically clear gain, across sequences, in HDR PSNR restricted to "seen only clipped" revealed pixels, together with a lower bound-violation rate. Overall PSNR must not drop by more than about 0.1 dB.
Why not just…?
Why not just rely on a saturating learned tone curve, as HDR-NSFF does? This is the most serious objection. A forward model that clips correctly already implements a one-sided loss: if predicted radiance exceeds the saturation point, rendering clips it to white and the loss is zero. Our expected advantage lies only in the interaction with the generative prior. HDR-NSFF's diffusion regularizer is not bounded, so wherever it is active it can drag radiance below a floor that the data already set. The second ablation (censoring without bracketed, projected inpainting) directly tests whether the forward model alone is enough. If it matches the full method on bound violations and target-subset PSNR, the honest conclusion is that clip-aware forward models suffice and the contribution reduces to an evaluation protocol.
Why not just inpaint in LDR and inverse-tone-map afterward? A single-exposure LDR fill cannot represent radiance above the chosen exposure's white point. Inverting the curve on saturated output only recovers the clip value. This is exactly the failure Gap 2 in the file describes for FreeOrbit4D-style pipelines. The cascade baseline measures how large the resulting loss is.
Why not just mask clipped pixels and let the inpainter decide? Masking throws away information the camera did record, such as "this window was brighter than X." Against the third ablation, our bound-violation metric quantifies how often an unconstrained inpainter contradicts that information. If violations are rare because inpainters already produce bright fills for windows, the gain will be small, and we will report that.
Why not just smooth per-frame metric depth over time? That is close to what we do for scale, and we make no novelty claim there. The experiment only asks whether picking the better-exposed frame of each pair beats plain temporal smoothing at stabilizing disocclusion masks.
Proposed Experiments
The data come from three sources. The first is the synthetic and real alternating-exposure benchmark of Mono4DGS-HDR. The second is HDR-GoPro, from which we build monocular alternating-exposure sequences by choosing frames across its bracketed synchronized cameras; the other cameras serve as held-out novel views. The third is a new set of Kubric-4D-style scenes we will render ourselves with linear HDR output and strong lighting, so that revealed surfaces with known ground-truth radiance exist. For real monocular capture, we follow the protocol of Monocular Dynamic View Synthesis: A Reality Check: smooth trajectories without camera "teleportation," and co-visibility-masked metrics. That paper's iPhone data, with LiDAR depth, cannot test HDR effects, but it can test the scale component.
Metrics include HDR PSNR on all pixels, on revealed pixels, and on revealed pixels seen only clipped, together with the bound-violation rate, temporal alignment error, and a ghost score. The ghost score is the residual correlation, in revealed regions, between rendered appearance and the departed object's appearance.
The decisive checks come in sequence. First, we measure the fraction of disoccluded pixels that are "seen only clipped" in real alternating-exposure sequences. If that fraction is negligible, the idea does not matter in practice, and we will say so before building further. Second, on the target subset, the full method must beat the HDR-NSFF-style recipe under the matched budget described in the baselines. Third, the projection ablation must account for most of that gain. Otherwise the benefit comes from bracketed inpainting alone, not from respecting the bounds.
References
- Mono4DGS-HDR: High Dynamic Range 4D Gaussian Splatting from Alternating-exposure Monocular Videos
- HDR-NSFF: High Dynamic Range Neural Scene Flow Fields
- Reconstruct, Inpaint, Test-Time Finetune: Dynamic Novel-view Synthesis from Monocular Videos
- FreeOrbit4D: Training-Free Arbitrary Camera Redirection for Monocular Videos via Geometry-Complete 4D Reconstruction
- ADNet: Attention-Guided Deformable Convolutional Network for High Dynamic Range Imaging
- DRHDR: A Dual branch Residual Network for Multi-Bracket High Dynamic Range Imaging
- Attention-Guided Progressive Neural Texture Fusion for High Dynamic Range Image Restoration
- Metric3D v2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimation
- Towards Zero-Shot Scale-Aware Monocular Depth Estimation
- RobustSplat: Decoupling Densification and Dynamics for Transient-Free 3DGS
- Monocular Visual-Inertial Depth Estimation
- Monocular Dynamic View Synthesis: A Reality Check