Abstract
Light Field (LF) imaging captures both the intensity and direction of light, enabling advanced applications such as digital refocusing and depth sensing. However, a physical resolution trade-off often limits the spatial quality of these captures. This paper introduces the Detail-Preserving Transformer (DPT), the first Transformer-based architecture specifically designed for Light Field Image Super-Resolution (LFSR). By treating LF views as sequences and employing a dual-branch architecture that incorporates gradient maps, DPT effectively captures long-range geometric dependencies and high-frequency details. Experimental results across five benchmarks demonstrate that DPT achieves state-of-the-art performance with high parameter efficiency.
Introduction
Light Field (LF) cameras provide a rich representation of 3D scenes but suffer from a fundamental trade-off between angular and spatial resolution. LFSR aims to computationally restore high spatial resolution by leveraging the rigid geometric structure between multiple viewpoints. While traditional Convolutional Neural Networks (CNNs) excel at local pattern recognition, they often struggle to model the global relationships and long-range dependencies across distant viewpoints in an LF array. DPT addresses this by redefining LFSR as a sequence-to-sequence reconstruction task, utilizing the strengths of self-attention mechanisms.
Methods
The DPT architecture is designed to overcome the "locality paradox" in super-resolution, where global context is necessary but local detail must be preserved. The core innovation is the Spatial-Angular Locally-Enhanced Self-Attention (SA-LSA) module, which uses convolutions to maintain local spatial context before applying self-attention to capture dependencies across view sequences.
As illustrated in Figure 1, the model features a dual-branch architecture:
- Content Transformer: Processes the original low-resolution images to capture primary scene information.
- Gradient Transformer: Processes image gradient maps, acting as a structural guide to force the model to prioritize high-frequency details and sharp edges.
These branches are integrated via a Fusion Transformer using cross-attention. To maintain computational efficiency, DPT processes horizontal and vertical sequences of views separately, effectively leveraging the Epipolar Plane Image (EPI) geometry of light fields.
Key Results
DPT was evaluated on five recognized LFSR benchmarks: EPFL, HCInew, HCIold, INRIA, and STFgantry. The model achieved superior performance, particularly at the challenging $4\times$ upscaling factor.
As shown in Figure 6, DPT outperforms previous leading methods like LF-DFNet. On the EPFL dataset ($4\times$ SR), DPT achieved a PSNR of 28.93 dB, compared to LF-DFNet's 28.77 dB. Remarkably, DPT is also highly efficient; for $4\times$ SR, it utilizes only 3.78M parameters, fewer than several competing CNN models. Qualitative analysis shows that DPT is significantly better at reconstructing complex, repetitive structures—such as telescope stands or fine edges—where CNNs often produce blurry artifacts or ghosting.
Conclusion
The Detail-Preserving Transformer demonstrates that sequence-based modeling is highly effective for Light Field Super-Resolution. By explicitly incorporating gradient priors and locally-enhanced attention, DPT sets a new standard for restoring 3D scene details. Future work may involve extending this architecture to 4D correlations or video light field applications.
Summary updated