A camera hidden under a phone screen looks at the world through a fine grid of display pixels. That grid acts as a diffraction grating: a lamp or the sun is smeared into long streaks and halos that can reach hundreds of pixels across the frame. The same lamp is also bright enough to saturate the sensor, so its core is clipped to the maximum value and its true brightness is lost. Current restoration networks handle this in one of two ways. Some treat the flare as a long-range problem and pay for global attention or whole-image Fourier transforms. Others treat the clipped core as something to hallucinate. We propose a third view, borrowed from erasure coding in distributed storage. The optical blur spreads each bright source redundantly over many distant pixels, much as an erasure code spreads a data symbol across many storage nodes. Clipping erases the core. The unclipped flare tail is then a set of "parity" measurements, and from them the lost brightness can be repaired.
The proposal has two parts. First, we estimate each clipped source's true radiance from a small, physically chosen set of unclipped tail pixels, using the measured point spread function (PSF), which describes how the display spreads a single point of light. Second, a restoration network does only local work for most of the image and gives long-range "helper" access only to the few pixels that lie inside a bright source's PSF footprint. The coding literature supplies a precise lens for the key empirical question: how many helper pixels, and how far away, are needed to repair an erased source to a given accuracy, and how many of those helpers can themselves be corrupted before repair fails. Cost should then scale with the number of highlights rather than with image area.
Core Claim
Under-display camera (UDC) degradation in high dynamic range (HDR) scenes is approximately
$$y(p) = \mathrm{clip}\Big(\sum_{s \in S} L_s\, k(p - s) + b(p) + n(p)\Big),$$
where $y(p)$ is the captured value at pixel $p$, $S$ is the small set of bright source locations, $L_s$ is the true (possibly clipped) radiance of source $s$, $k$ is the display PSF, $b(p)$ is the locally blurred and scattered remainder of the scene, $n$ is noise, and $\mathrm{clip}$ saturates at the sensor maximum. This extends the convolution model of Image Restoration for Under-Display Camera with explicit clipping and source sparsity.
Because $k$ has long, spiky sidelobes, which DISCNet reports to exceed $800 \times 800$ pixels, each $L_s$ appears linearly in many unclipped pixels far from its core. Given a helper set $R_s$ of such pixels, a robust estimate is
$$\hat L_s = \arg\min_{L}\; \sum_{p \in R_s} \rho\big(y(p) - L\,k(p-s) - \hat b(p)\big),$$
where $\rho$ is a robust loss and $\hat b$ is a local estimate of the background. Our claim is that a small, PSF-chosen $R_s$ recovers $L_s$ accurately. Once $\hat L_s$ is known, the long-range flare can be subtracted explicitly. The network then only needs local context plus a handful of source tokens.
We expect this to beat purely learned models in highlight regions at lower cost, and to make the recovered highlight radiance measurable rather than hallucinated. The storage-coding results are not imported as theorems. The generalized Singleton bound of On the Locality of Codeword Symbols concerns finite-field linear codes and does not transfer literally. What transfers is the design question it poses: repair locality versus tolerance to corrupted helpers. We measure that tradeoff empirically.
Relation to Prior Work
Four UDC methods motivate the proposal and set the bar. Each shows that spatially adaptive processing matters, and none treats the flare tail as information about the clipped core.
DISCNet encodes the measured PSF and uses it to drive dynamic convolutions in skip connections. It identifies saturation as breaking shift-invariance but handles it implicitly. UDC-UNet modulates features by local intensity using spatial feature transform (SFT) layers. SFT is a per-pixel scale-and-shift of network activations predicted from a condition signal. UDC-UNet thereby suppresses flare strongly near highlights and gently in shadows. Its long range comes from a U-shaped multi-scale hierarchy, not from any source model. We reuse its intensity-conditioned modulation, but we condition on an explicit predicted flare map built from $\hat L_s$ and $k$.
SGSFormer finds that dense attention spreads haze and noise into clean regions, and it restricts attention using masks from the Segment Anything Model. This result constrains our design: unrestricted long-range mixing is harmful. We agree, but we choose neighborhoods from the optical geometry rather than from semantic masks, and we avoid running a large segmentation model.
FSI exploits the fact that periodic diffraction is sparse in frequency and filters it with whole-image Fourier transforms. That approach is both global in memory and, by the file's account, degraded when clipping spreads energy non-periodically. SRUDC adds a scattering veil estimated by global attention and fused through an affine block. We keep a scattering term inside $b(p)$ but estimate it locally.
From storage coding we borrow two ideas. The first is the pyramid structure analyzed by Gopalan et al.: small local repair groups plus a few global parities. In our setting the local window plays the role of the local group and the source tokens play the role of global parities. The second is unequal locality from Codes with Unequal Locality, which gives different symbols different repair budgets. In our setting, pixels inside a PSF footprint get wider helper sets and background pixels get only local ones. XORing Elephants is the systems precedent for trading a little extra redundancy for much cheaper repair. We use it only as a framing.
Research Program
The first stage is a measurement study, and it can falsify the whole idea cheaply. On HDR data with known ground-truth radiance, we simulate clipping and solve the robust estimator above. We use helper sets drawn along the PSF's strongest sidelobes at varying count $r$ and distance from the core. We then plot radiance error against $r$ and against the fraction of helpers contaminated by other sources, texture, or noise. This yields an empirical locality-versus-tolerance curve: the minimum $r$ for a target error, and how much contamination a given $r$ survives. We also measure how this curve moves under PSF perturbations (small rotations, scalings, and aperture changes), since panel-to-panel variation is a known open problem for PSF-conditioned methods.
The second stage builds the network. A cheap detector finds saturated cores and proposes source tokens. A small module refines $\hat L_s$, initialized from the physics estimate and trained end to end. From $\hat L_s$ and $k$ we render an explicit flare map, subtract it, and also feed it as the condition for SFT modulation in a windowed local backbone. Each pixel attends to its local window. It also attends to source tokens only if its predicted flare exceeds a threshold, which implements unequal locality. The helper budget per pixel is a tunable knob. Sweeping that knob traces a compute-versus-quality curve for the network, which mirrors the stage-one curve.
Baselines
The naive reference point is classical PSF deconvolution with denoising. Image Restoration for Under-Display Camera reports 28.50 dB PSNR on T-OLED and 16.97 dB on P-OLED for this approach, against 36.71 and 30.45 dB for a learned UNet. We rerun it on our HDR benchmarks. We also include a clip-aware variant that uses only our physics radiance estimate followed by flare subtraction, with no network. This variant isolates how much the "parity" estimate does by itself.
The strongest learned competitor is UDC-UNet, which combines intensity-conditioned modulation with multi-scale long range. It took second place in the MIPI 2022 UDC challenge, where the second-place XPixel Group entry reports 47.78 dB PSNR at 0.16 s per image on an A6000 (MIPI 2022 challenge report). The first-place entry reports 48.48 dB, and we treat it as the headline number to approach. We retrain UDC-UNet and its efficient 169-GMAC variant on identical splits, losses (including its tone-mapped L1), and training budget.
We also run DISCNet, which reports 43.27 dB on its synthetic HDR set, as the strongest PSF-conditioned dynamic-convolution model. We run FSI as the global-frequency competitor, which reports 46.14 dB on SYNTH. SGSFormer is the sparse-attention competitor; its reported 39.10 dB on T-OLED is not an HDR setting.
Beating UDC-UNet means matching its full-image PSNR at no more than its MACs, while improving highlight-region PSNR and linear radiance error in clipped regions by a margin larger than seed variance.
Two ablations isolate the key ingredient. The first replaces physics-chosen source tokens with an equal number of learned global tokens. The second removes the explicit flare map while keeping the source tokens.
Why not just…?
Why not just rely on the U-Net bottleneck, which already sees 800 pixels? This is the real threat. UDC-UNet's multi-scale hierarchy reaches the whole flare footprint and already scores above DISCNet and Uformer by over 7 dB. A bottleneck sees the flare only at low resolution, though, and nothing forces it to infer the clipped radiance consistently with the PSF. The matched-MAC comparison, scored on highlight regions and on radiance error, settles this. If UDC-UNet matches us there, the parity framing adds nothing and we will say so.
Why not just do clip-aware deconvolution with the measured PSF? Classical deconvolution collapses on real panels, as Zhou et al.'s numbers show. Its failure, however, involves noise and scattering as well as clipping. We include the physics-only estimator as a baseline because, if it alone recovers most of the gain, the network is unnecessary.
Why not just add a few learned global tokens to a windowed transformer? Generic global tokens give long range at low cost. They do not know where the sources are or which pixels carry information about each source. The learned-token ablation answers this directly. Physics-placed tokens should win most clearly when there are several overlapping highlights.
Why not just filter in the frequency domain like FSI? Whole-image Fourier filtering needs the full frame in memory and is disturbed by clipping. We compare peak memory and highlight-region quality at increasing resolution.
Proposed Experiments
The primary benchmark is MIPI 2022. It is an HDR set of 800×800 pairs synthesized from HDR panoramas with measured PSFs, so ground-truth radiance exists and clipped inputs can be generated; we will verify the clipping protocol. We also use the DISCNet synthetic HDR set with real ZTE Axon 20 captures for qualitative and no-reference checks, and the SRUDC dataset to test the scattering term. T-OLED and P-OLED serve as low-dynamic-range sanity checks, where we expect parity with UDC-UNet rather than gains.
Metrics are tone-mapped PSNR, SSIM, and LPIPS. We add PSNR restricted to PSF footprints of saturated sources, relative error of recovered linear source radiance, MACs, and peak memory at 800×800 and at upsampled 4K tiles.
The decisive checks are three. First, the stage-one curve must show radiance error below about 10% with tens rather than thousands of helpers; otherwise the idea is dead. Second, the network must beat retrained UDC-UNet on highlight metrics at matched MACs. Third, cost must grow with the number of sources, not with image area. A further robustness test perturbs the PSF at test time to see whether the explicit estimator degrades more gracefully than DISCNet-style implicit conditioning.
References
- Image Restoration for Under-Display Camera
- Removing Diffraction Image Artifacts in Under-Display Camera via Dynamic Skip Connection Network
- On the Locality of Codeword Symbols
- UDC-UNet: Under-Display Camera Image Restoration via U-shape Dynamic Network
- Segmentation Guided Sparse Transformer for Under-Display Camera Image Restoration
- FSI: Frequency and Spatial Interactive Learning for Image Restoration in Under-Display Cameras
- Under-Display Camera Image Restoration with Scattering Effect
- Codes with unequal locality
- XORing Elephants: Novel Erasure Codes for Big Data
- MIPI 2022 Challenge on Under-Display Camera Image Restoration: Methods and Results