TL;DR
Loop-Mamba is a lightweight, loop-based state-space framework designed to restore old photographs by treating the process as a progressive state evolution rather than repeated image reconstruction. By integrating a semantic-guided degradation estimator with a shared structural memory, it effectively addresses complex, mixed corruptions while maintaining linear computational complexity. The network achieves state-of-the-art restoration quality and significantly faster inference speeds, making real-time processing and mobile deployment feasible.
What Problem Does This Paper Address?
Old photographs typically suffer from a combination of severe, overlapping degradations—such as scratches, cracks, fading, blur, and completely missing regions. Restoring these images is an inherently ill-posed problem because these heterogeneous corruptions heavily damage the underlying structural information of the image.
Existing restoration paradigms face a tough trade-off:
- CNN and Transformer-based iterative methods progressively refine images but rely on repeatedly stacking deep feature transformations or computing heavy global attention. This leads to severe computational overhead and long gradient propagation paths that cause gradient dilution.
- Standard Mamba/State Space Models (SSMs) offer linear complexity but typically lack mechanisms to explicitly model mixed degradations or maintain a persistent memory of recovered structures across iterations.
Loop-Mamba aims to bridge this gap by providing an efficient, loop-based framework that explicitly models degradation severity and continuously propagates structural memory across iterations without exploding the computational budget.
Methods
The architecture of Loop-Mamba relies on three core components that transform image restoration into a first-order state recursion process:
- Semantic-Guided Degradation Estimator (SGDE): Instead of treating all regions equally, the SGDE uses a CLIP semantic prior to jointly predict a pixel-level local degradation map (where the damage is) and an image-level global degradation score (how severe the overall damage is). This representation guides the subsequent state evolution.
- Shared Structural Memory Mamba ($S^2$M-Mamba): Rather than reconstructing features from scratch at each loop iteration, Loop-Mamba maintains a persistent hidden state ($M_t$) across iterations. This shared memory continuously accumulates and fuses structural representations, ensuring long-range structural consistency.
- Multi-Directional State Propagation: To enhance spatial context processing, the framework incorporates a lightweight scanning strategy that propagates states along four distinct spatial directions, enriching feature interaction with negligible overhead.
Optimized end-to-end, the network combines standard reconstruction losses ($L_1$, SSIM, edge, and Total Variation) with a dedicated degradation supervision loss.
Datasets
The authors evaluated Loop-Mamba using both synthetic and real-world datasets:
- SynOld Benchmark: A public dataset consisting of 700 image pairs (500 for training, 50 for validation, and 200 for testing) used for quantitative and qualitative evaluation against full-reference metrics.
- Stockfilm Dataset: Comprising 500 frames extracted from various real-world film stocks, utilized to test the model's generalization capabilities on real-world old photo degradations.
To better evaluate performance, the authors also introduced a new task-oriented metric: the Old Photo Damage Recovery Score (ODRS), which mathematically combines a localized Damage Recovery Score (DRS) with a CLIP-based Structural Recovery Score (SRS).
Key Insights and Ideas
- First-Order State Recursion vs. Deep Stacking: By formulating the loop as a first-order recurrence relation ($St+1 = ASt + BXt$), the gradient propagation path length remains constant regardless of the number of iterations. This completely bypasses the gradient dilution problem common in deep recurrent CNNs.
- Decoupling Degradation from Restoration: Explicitly estimating the degradation map using semantic priors allows the restoration backbone to adaptively focus its computational capacity on severely damaged regions.
- Linear Complexity Efficiency: Leveraging the linear computational complexity of State Space Models allows Loop-Mamba to perform iterative refinement at a fraction of the computational cost of Transformer-based alternatives.
Key Results
Loop-Mamba achieves a superior quality-efficiency trade-off compared to CNN, Transformer, Diffusion, and other Mamba-based architectures:
- Quantitative Performance: On the SynOld benchmark, Loop-Mamba achieved the highest scores across major metrics, reaching a PSNR of 30.13 dB, an SSIM of 0.93, and an ODRS of 0.9495.
- Computational Efficiency: When processing 1080p images, Loop-Mamba required only 119 GFLOPs and 59 GMACs, outperforming competitive models like Restormer (4568 GFLOPs) and MPRNet (34719 GFLOPs). It achieved an inference speed of 1.86 FPS, the highest among the benchmarked iterative methods.
- Mobile Deployment: The authors successfully deployed the model on a commercial smartphone (Mi 10 Lite Zoom with an ARM64 processor) using ONNX Runtime, proving its viability for edge-device applications.
Limitations and Future Work
The authors observed a performance threshold regarding the number of restoration cycles ($t$). While increasing the iterations from 1 to 3 steadily improves image quality metrics, performance gradually declines if $t$ is increased beyond 3 (e.g., at $t=4$ or $t=5$). This suggests that excessive iterations can lead to over-refinement or artifact generation.
Future work could explore adaptive iteration strategies where the model dynamically decides when to stop the loop based on real-time quality assessment of the evolving state.
Summary updated