MaskBlur: Spatial and Angular Data Augmentation for Light Field Image Super-Resolution

Wentao Chao, Fuqing Duan, Yulan Guo, Guanghui Wang

Introduction

Light Field (LF) cameras capture rich 4D information by recording both the spatial position and angular direction of light. This capability enables advanced applications like post-capture refocusing and depth estimation. However, a fundamental hardware trade-off often results in high angular resolution at the expense of spatial resolution. While deep learning has significantly advanced LF super-resolution (SR), these models typically require vast amounts of high-quality data, which is labor-intensive to collect. Existing data augmentation (DA) techniques often fail to respect the unique 4D geometry of light fields, either focusing solely on spatial consistency (e.g., CutBlur) or disrupting spatial structures through angular blending (e.g., CutMIB).

The MaskBlur Framework

To address these limitations, we introduce MaskBlur, a novel data augmentation strategy designed specifically for LF images. MaskBlur concurrently addresses the spatial and angular domains through two primary components:

  1. Spatial Blur: Unlike traditional rectangular cropping, MaskBlur employs a patch-based random spatial mask. It selectively swaps pixels between low-resolution (LR) and high-resolution (HR) domains. This approach forces the model to handle varying degrees of sharpness and blur within a single image, enhancing its ability to resolve fine spatial details.
  2. Angular Dropout: Inspired by standard Dropout, this component uses a random angular mask to determine which specific viewpoints (Sub-Aperture Images or SAIs) receive the spatial blur operation. By preventing the model from treating all views identically, it increases the diversity of angular information available during training.

Crucially, MaskBlur uses small "blocks" as the masking unit rather than individual pixels, which preserves local information completeness and maintains the underlying edge structure essential for high-quality SR.

Experimental Setup

The effectiveness of MaskBlur was evaluated using five widely recognized LF datasets: EPFL, HCInew, HCIold, INRIA, and STFgantry. Real-world performance was assessed using the STF Lytro archive and the NTIRE 2024 LFSR validation set. Models were primarily evaluated on the luminance (Y) channel using Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM).

Key Results

MaskBlur was integrated into several state-of-the-art LF-SR models, including ATO, IINet, DistgSSR, and EPIT.

Performance comparison of MaskBlur against other data augmentation methods across four LF-SR models.

As shown in the figure above, MaskBlur consistently outperforms baseline models and other augmentation strategies like CutBlur and CutMIB. For instance, it improved the average PSNR of the ATO model by 0.378 dB on the HCIold dataset. Even for highly optimized architectures like EPIT, MaskBlur provided a measurable performance boost.

Beyond super-resolution, MaskBlur's "mixed-quality" training makes models more robust for other restoration tasks. In high-noise denoising scenarios (σ=50), the strategy improved the PSNR of the InterNet model by 0.110 dB, resulting in significantly clearer textures and sharper lines in qualitative assessments.

Conclusion

MaskBlur demonstrates that concurrent spatial and angular augmentation is key to learning robust 4D light field representations. The method is computationally efficient, adding negligible overhead during training and zero overhead during inference. While currently focused on static light fields, future research could extend this framework to dynamic scenes or video-based light fields, where temporal consistency presents an additional dimension for augmentation.

Summary updated

BibTeX
@article{chao2025maskblur,
  author = {Wentao Chao and Fuqing Duan and Yulan Guo and Guanghui Wang},
  title = {{MaskBlur: Spatial and Angular Data Augmentation for Light Field Image Super-Resolution}},
  journal = {IEEE Trans. Multimedia},
  year = {2025},
  doi = {10.1109/tmm.2024.3521781},
}
Figure