TL;DR
DeLang-SR introduces a one-step flow-matching framework for low-light super-resolution that solves the problem of "one-size-fits-all" restoration. By explicitly decoupling the direction of restoration (steered by structured degradation language) from its magnitude (controlled by a spatially varying update-strength field), it achieves state-of-the-art results in brightening and sharpening dark images while avoiding the artifacts typically caused by uniform global updates.
What Problem Does This Paper Address?
Super-resolution in the dark is a nightmare for standard generative models because of spatial heterogeneity. In a single low-light photo, some regions might be pitch black and buried in noise, requiring aggressive reconstruction, while other regions (like those near a streetlamp) are relatively well-exposed and only need a light touch.
Standard diffusion and flow-matching models typically apply a globally uniform restoration trajectory. This creates a "mismatch": if the model is too aggressive, it hallucinates textures in reliable areas; if it’s too conservative, it fails to recover details from the shadows. The authors argue that we need a way to tell the model not just what to fix, but how much to fix at every single pixel.
Methods
DeLang-SR is built upon the Stable Diffusion 3 (SD3) flow-matching architecture, adapted via parameter-efficient LoRA modules. Its core innovation lies in the decoupling of two factors:
- Direction Steering (The "What"): The model translates measurable physical statistics (like exposure gap, shadow noise, and color shift) into structured degradation language. For example, it might generate a prompt like: "Restore a real low-light image with severe underexposure and high shadow noise." This semantic intent is combined with visual degradation tokens (local features like luminance and contrast) to steer the velocity field toward a high-quality image manifold.
- Magnitude Control (The "How Much"): Instead of a uniform step, the model predicts a Spatial Update-Strength Field ($M_\phi$). This field acts as a locally varying Euler step-size map. In dark, noisy regions, $M_\phi$ is high, allowing for a large corrective leap. In reliable regions, $M_\phi$ is low, enforcing a conservative update that preserves original details.
The final restoration is performed in a single step, making it significantly more efficient than iterative diffusion processes.
Datasets
The authors evaluated DeLang-SR across several benchmarks:
- RELLISUR: A primary paired dataset specifically designed for joint low-light enhancement and super-resolution.
- DarkFace: A collection of real-world nighttime scenes used for no-reference evaluation to test generalization.
- LOL-v2-Real & LSRW: Used for cross-dataset transfer experiments to ensure the model isn't just "overfitting" to one type of darkness.
Key Insights and Ideas
- Decoupling is Essential: By separating the direction of the flow from the magnitude of the update, the model can handle the extreme contrast of low-light scenes without needing complex cascaded pipelines.
- Language as a Physical Interface: Rather than relying on manual prompts, the system automatically maps raw image statistics to natural language. This allows the model to tap into the massive generative priors of text-to-image models (like SD3) using a familiar "interface."
- Interpretable Restoration: The update-strength map ($M_\phi$) provides a "heat map" of where the model is working hardest. This makes the black-box nature of generative restoration much more transparent.
Key Results
DeLang-SR consistently outperformed both deterministic backbones (like Restormer and HAT) and recent generative methods (like DHGM and DOD).
- Quantitative Gains: On the RELLISUR $\times 2$ task, DeLang-SR achieved a PSNR of 23.865, a significant jump over the previous best generative method, DHGM (22.663). It also led in perceptual metrics like LPIPS and DISTS.
- Efficiency: Despite being based on a large SD3 model, the one-step approach allows for inference times (~0.44s) that are competitive with or faster than many multi-step diffusion models.
- Qualitative Superiority: Visual results show that DeLang-SR preserves sharper structures and more faithful colors in "extreme dark" regions where other models produce blurry or color-shifted results.
Limitations and Future Work
While powerful, DeLang-SR still carries the computational footprint of a large pretrained transformer (SD3). While the one-step inference is fast, the memory requirements remain high. Additionally, in "pitch black" regions with zero signal, the model may still hallucinate plausible but technically "fake" details—a common trait of all generative restoration.
The authors suggest future work could explore lightweight flow priors to reduce the model size and uncertainty-aware estimation to better calibrate the update-strength field in ambiguous regions.
Summary updated