A light field camera records each scene point from many slightly shifted viewpoints. Because the sensor has a fixed number of pixels, every extra viewpoint costs sharpness in each view. This is the spatial-angular resolution trade-off, and light field super-resolution tries to buy the sharpness back computationally by pooling evidence across views. The best current networks do this with attention along epipolar plane images (EPIs). An EPI is a 2D slice of the light field in which each scene point traces a straight line whose slope equals its disparity. At a depth edge that geometry breaks down: in some views a background point is hidden behind a foreground object. An attention layer that reads the background point's trajectory therefore reads foreground pixels, and it mixes them into the background reconstruction.
Text-to-image generation ran into the same failure under a different name, "attribute leakage," and fixed it by writing an additive mask directly into the attention scores so that tokens cannot draw information across region boundaries. We propose to carry that fix over to light fields in the one form the geometry actually justifies. The mask is a soft, learned-strength penalty that stops a query pixel from attending to keys that belong to a nearer surface occluding the query's own trajectory. The mask is computed from a coarse disparity map and its confidence, and a separate learned strength is applied in each layer. That per-layer strength lets the network keep global attention wherever it helps with reflections and transparency.
Core Claim
Occlusion information helps light field super-resolution more when it enters as a depth-ordered bias on the attention scores than when it enters as an extra input feature. The gain should concentrate in pixels near occlusion boundaries. Leaving the mask unapplied in some layers should matter on non-Lambertian scenes, meaning scenes with reflections, specular highlights or transparency, whose apparent disparity does not match the surface depth.
The closest prior work falls into three roles. Occlusion-Embedded Hybrid Transformer for Light Field Super-Resolution (OHT) motivates the proposal. It computes per-view occlusion maps from photometric consistency and a coarse disparity, but fuses them through a convolutional mix layer, so its angular self-attention remains unmasked. OPAL masks occluded views, but only inside an unsupervised loss for disparity estimation, never inside its EPI-Transformer. ECSIC restricts stereo cross-attention to epipolar lines, which is a correspondence mask with no notion of which surface is in front. The masking technique itself comes from Localized Text-to-Image Generation for Free via Cross Attention Control, DenseDiffusion and Seg2Any. Seg2Any constrains our design with a negative lesson: masking every layer destroys global coherence, and masking works best in a band of middle layers. Our mask differs from all of these in one respect: it penalizes keys according to depth order along a geometric trajectory, rather than according to region membership or line membership.
Research Program
We build on EPIT. EPIT alternates self-attention over horizontal and vertical EPIs, has about 1.47M parameters, and was shown to be robust to large disparities. Consider one EPI with angular coordinate $u$ and spatial coordinate $x$. Let $d$ be a coarse disparity map with confidence $c$, both taken from a light disparity head supervised on synthetic data. We add a bias $M^{(l)}$ to the attention logits of layer $l$:
$$M^{(l)}_{ij} = -\alpha_l \, c_j \, \sigma\!\left(\frac{d_j - d_i - \tau}{s}\right) \exp\!\left(-\frac{\left(x_j - x_i - d_i (u_j - u_i)\right)^2}{2 w^2}\right)$$
The symbols are as follows. $i$ is the query pixel and $j$ is the key pixel. $\sigma$ is the logistic function, $\tau$ is a disparity margin and $s$ is its softness. $w$ is the band half-width around the query's predicted trajectory. $\alpha_l \ge 0$ is the strength for layer $l$; it is learned and initialized to zero, so training starts from plain EPIT.
The exponential term selects keys that lie on the query's own line through the EPI. The logistic term fires only when the key is nearer to the camera than the query, that is, when it is an occluder covering the query's ray in that view. Keys off the trajectory are untouched. The global receptive field that gives EPIT its robustness is therefore preserved.
Because the mask is soft and weighted by confidence, errors in the disparity estimate degrade it gracefully. This matters because disparity is least reliable exactly at the boundaries where the mask is needed. The learned values of $\alpha_l$ also serve as an analysis tool. If they settle into a middle band of layers, that reproduces Seg2Any's finding in a new domain.
The work has three stages. The first is a diagnostic done before building anything. On the HCI benchmark, which has ground-truth disparity, we measure how much attention mass a trained EPIT places on occluder keys for background queries near edges. The second stage trains the masked model with estimated disparity, and also with an oracle ground-truth-disparity mask as an upper bound. The third stage analyzes the learned $\alpha_l$ profiles and relates them to per-scene non-Lambertian content, such as the refractive and reflective scenes in the MPI archive.
Baselines
The naive reference points are bicubic upsampling and a single-image super-resolution network applied independently to each view. Light Field Super-Resolution: A Benchmark showed that VDSR, one such network, can beat older light field methods. No reported result was found on Lacuna for VDSR on BASiCLFSR, so we will retrain it on the same 144 training scenes.
The first strongest competitor is unmodified EPIT, since our model is EPIT plus a bias. The file reports that EPIT improves PSNR on STFgantry over the prior best method (LFT) by 0.32 dB at ×4 and 1.66 dB at ×2 (EPIT). We retrain EPIT with our code, schedule and seeds.
The second strongest competitor is OHT-B. The file reports it as the highest in PSNR and SSIM on BASiCLFSR at ×2 and ×4, ahead of DET by 0.08 dB at ×2 (OHT). No absolute OHT-B PSNR figure appears in the file. For absolute context on the related NTIRE test set, the NTIRE 2025 classical-track winner reports 31.16 dB average PSNR (NTIRE 2025). That winner relies on masked pre-training, a separate ingredient that we could stack on top of our mask but do not compete with.
The decisive ablation keeps our disparity head but feeds its occlusion maps into EPIT through an OHT-style convolutional mix layer instead of the bias. The two arms then use identical information and differ only in where that information enters. A second ablation matches the parameter count with a learned relative-position bias that has no geometry, to show that improvement comes from depth order and not from extra flexibility.
All arms share the BASiCLFSR split, 5×5 views, iterations, and a hyperparameter search budget of equal size. We report a mean over three seeds. We count the proposal as beating the baselines if it outperforms the input-concatenation ablation and retrained EPIT in two ways: in mean PSNR on BASiCLFSR at ×4, and by a larger margin in boundary-band PSNR, with gaps exceeding the variance across seeds.
Why not just…?
Why not just concatenate the occlusion map as input, as OHT does? This is the most serious threat. OHT's reported margin over DET is small, and that is consistent with input-level occlusion cues being diluted by the time they reach the attention layers. It does not prove the point, however. The same-information ablation settles the question directly. If input concatenation matches the bias in boundary-band PSNR, the core claim is false and we will report it as such. The 2D generation results, in which mask conditioning alone still leaked and score modulation raised MS-COCO IoU from 23.76 to 34.99 (DenseDiffusion), make us expect otherwise, but they come from a different task.
Why not just let attention learn to ignore occluders on its own? It may already do so. The stage-one diagnostic tests this before we build anything. If EPIT's attention mass on occluder keys is already negligible compared with its mass on the query's own surface, the mask has nothing to fix and we stop.
Why not just warp neighboring views with estimated depth? Explicit warping fails at occlusion boundaries and on non-Lambertian surfaces, as the LFdiff study reports (Diffusion-based Light Field Synthesis). Non-local attention beat depth-warping methods on the MPI data by more than 5 dB (40.81 dB versus 35.15 dB for 16× angular upsampling, SAA-Net). Our mask uses depth only to down-weight keys, never to move pixels, so a wrong depth value costs less than it would under warping.
Why not just use rigid epipolar-line attention? Restricting attention to the line, as ECSIC does, tells the model which pixels correspond but not which surface is visible. The background trajectory still passes through foreground pixels. We include an ECSIC-style hard band mask, meaning the trajectory term alone without the depth-order term, as an additional arm.
Proposed Experiments
The main evaluation is BASiCLFSR at ×2 and ×4, reporting PSNR and SSIM per subset (EPFL, HCInew, HCIold, INRIA, STFgantry). We also report LPIPS so the results are comparable to LFSRDiff, which reports an average of 0.1392.
The decisive check uses HCI ground-truth disparity. We split test pixels into an occlusion-boundary band and an interior region, and report PSNR for each. The claim predicts that the gain over both EPIT and the concatenation ablation sits mostly in the band.
Three further checks probe robustness and the per-layer design. The oracle-disparity arm measures how much of the effect is lost to estimation error. EPFL, captured with a real plenoptic camera, tests whether noisy disparity erases the benefit, given the known drop of about 1.25 dB from synthetic to real data. On MPI's reflective and refractive scenes, we compare the learned per-layer $\alpha_l$ against a variant that forces the same strength in every layer. If the per-layer variant does not win there, the claim about protecting global attention for non-Lambertian effects fails.
References
- Occlusion-Embedded Hybrid Transformer for Light Field Super-Resolution
- OPAL: Occlusion Pattern Aware Loss for Unsupervised Light Field Disparity Estimation
- ECSIC: Epipolar Cross Attention for Stereo Image Compression
- Localized Text-to-Image Generation For Free via Cross Attention Control
- Dense Text-to-Image Generation with Attention Modulation
- Seg2Any: Open-set Segmentation-Mask-to-Image Generation with Precise Shape and Semantic Control
- Learning Non-Local Spatial-Angular Correlation for Light Field Image Super-Resolution
- Light Field Super-Resolution: A Benchmark
- NTIRE 2025 Challenge on Light Field Image Super-Resolution: Methods and Results
- Diffusion-Based Light Field Synthesis
- Spatial-Angular Attention Network for Light Field Reconstruction
- LFSRDiff: Light Field Image Super-Resolution via Diffusion Models