TL;DR
Enhancing ultra-high-definition (4K/8K) low-light images in real-time is notoriously difficult due to the massive computational and memory footprints of modern networks. This paper introduces Clifford Pyramid Enhance (CPE), a framework that leverages 2D Euclidean Clifford Algebra to efficiently fuse high- and low-frequency image features within a low-resolution latent space before mapping them back to full resolution. The result is a state-of-the-art enhancement method that achieves blazing-fast, millisecond-level inference (over 115 FPS at 4K and 87 FPS at 8K) on a single consumer GPU without hitting the "memory wall."
What Problem Does This Paper Address?
As 4K and 8K resolutions become standard in applications like autonomous driving and surveillance, processing these massive pixel arrays in low-light conditions poses a severe challenge. Existing state-of-the-art models, such as Transformers or diffusion models, achieve excellent image quality but suffer from a "memory wall" bottleneck. Their quadratic computational complexity leads to Out-of-Memory (OOM) errors on standard hardware.
To avoid OOM errors, practitioners often resort to patch-based processing or global downsampling. Unfortunately, patch-based methods introduce unnatural boundary artifacts ("grid-like patch seams"), while downsampling permanently destroys fine, high-frequency textures. On the flip side, ultra-lightweight curve-estimation models are fast but lack noise robustness, frequently amplifying background noise and causing severe color washout. The authors aim to bridge this gap by designing an architecture that delivers both high-fidelity physical restoration and real-time processing speeds for native UHD inputs.
Methods
The authors propose Clifford Pyramid Enhance (CPE), which elegantly decouples heavy feature extraction from full-resolution image reconstruction:
- Inverse Feature Pyramid & Frequency Decoupling: The network downsamples the native UHD input to a compact base tensor (e.g., 64×64) and progressively upsamples it through a multi-scale pyramid. At each level, a Gaussian blur kernel splits the features into low-frequency (global illumination) and high-frequency (structural details) components. These are processed by parallel, lightweight U-Nets (
LightUNet) using depthwise separable convolutions to keep parameters minimal. - Clifford Geometric Feature Fusion: Instead of traditional linear concatenation or addition—which destroys geometric topology and mixes noise with texture—the authors introduce 2D Euclidean Clifford Algebra, $Cl(2,0)$. Features are mapped into a multivector space containing scalars, vectors, and bivectors. By calculating a spatial correlation mask using the Clifford inner product, the network preserves sharp structural edges where gradients align, while naturally suppressing random, directionless noise in dark regions.
- Dynamic Resolution Reconstruction: To avoid OOM errors, heavy convolutions are kept strictly within the low-resolution latent space. The fused multi-scale features are upsampled and concatenated to predict spatially adaptive Gamma ($\Gamma$) and Gain ($G$) maps at the native resolution. Guided by Retinex theory, these maps adjust the initial illumination component non-linearly, ensuring proportional scaling across RGB channels to maintain strict color constancy.
Datasets
The model was trained and evaluated on the authoritative UHD-LOL dataset, which contains real-world low-light and normal-light image pairs:
- 4K Subset: 5,999 pairs for training and 2,100 pairs for testing.
- 8K Subset: 2,029 pairs for training and 937 pairs for cross-scale stress testing.
Key Insights and Ideas
- Bypassing the Memory Wall via Parameter Mapping: Instead of generating massive RGB pixel arrays directly, the network predicts lightweight, resolution-agnostic transformation parameters (Gamma and Gain maps) at full resolution. This keeps the heavy feature extraction confined to a tiny latent space, completely avoiding OOM errors.
- Clifford Algebra as an Attention Alternative: Traditional self-attention mechanisms are computationally prohibitive for UHD images. CPE uses the geometric properties of Clifford algebra to capture spatial directionality and cross-channel correlations at a fraction of the computational cost.
- Physics-Grounded Color Constancy: By enforcing the constant reflectance assumption from Retinex theory and applying a single illumination ratio across all RGB channels, the model mathematically guarantees that enhanced images remain free of the color shifts common in standard deep learning approaches.
Key Results
- Blazing Speed and Low Memory: CPE achieves an average latency of 8.68 ms (115.20 FPS) at 4K and 11.49 ms (87.00 FPS) at 8K on a single NVIDIA RTX 3090 GPU, consuming only 1.22 GB of VRAM at 8K. In comparison, heavy models like LLFormer take over 160 seconds per 8K image.
- High Reconstruction Quality: At 8K resolution, CPE delivers a PSNR of 22.86 dB and an SSIM of 0.9453, outperforming downsampling-based and lightweight native baselines.
- Downstream Task Performance: When evaluated on a downstream object detection task (using a pre-trained YOLOv10x without fine-tuning), CPE-enhanced 8K images allowed the detector to find 576 targets, vastly outperforming other real-time baselines while achieving a 14,000× speedup over heavyweight alternatives.
- Edge Deployment: The authors successfully deployed an FP16 mixed-precision version on commercial smartphones. On a Huawei Mate 60 Pro, the pure NPU execution took just ~279 ms for an 8K image, demonstrating true edge-deployment viability.
Limitations and Future Work
- Color Halos: Because the model relies heavily on Retinex illumination priors, it can occasionally produce slight color halos around intense point light sources, such as vehicle high beams in nocturnal scenes.
- Video Flickering: When directly applied frame-by-frame to UHD video streams, the single-frame architecture may exhibit minor temporal flickering due to a lack of inter-frame constraints.
- Future Directions: The authors plan to explore generative priors to handle extreme "blind spots" and intend to introduce 3D spatio-temporal geometric manifolds to optimize temporal consistency in dynamic video streams.
Summary updated