Hierarchical Bilevel Learning with Architecture and Loss Search for Hadamard-based Image Restoration

Guijing Zhu, Long Ma, Xin Fan, Risheng Liu

Abstract

This paper presents a Hierarchical Bilevel Learning framework designed to automate the discovery of optimal network architectures and loss functions for image restoration tasks. By incorporating a "Hadamard-inspired" neural unit that leverages physical domain knowledge of image degradation, the framework addresses the limitations of manual tuning and generic neural architecture search. Evaluated on low-light enhancement, dehazing, and underwater restoration, the proposed method achieves state-of-the-art performance across multiple benchmarks.

Introduction

Image restoration often requires painstaking manual tuning of both network architectures and loss functions. While Neural Architecture Search (NAS) has automated parts of this process, it frequently overlooks the physical principles governing image degradation, such as the multiplicative effects of haze or low light. Furthermore, standard loss functions like Mean Squared Error ($L_2$) often fail to align with human visual perception, leading to blurry outputs. This research introduces a system that simultaneously searches for the most effective architecture and loss combination for specific restoration tasks.

Methodology

The core of the proposed framework lies in three primary innovations: the Hadamard-inspired Neural Unit (HNU), a Triple-Level Optimization strategy, and a Hierarchical Bilevel Solver.

The HNU explicitly models multiplicative degradation ($y = u \otimes v$), where a degraded image $y$ results from a clear image $u$ and a degradation factor $v$. This approach allows the network to "unroll" the degradation process based on physical priors rather than acting as a generic black box.

The proposed architecture featuring the Feature Convert Module (FCM) and Hadamard-inspired Neural Unit (HNU) within an encoder-decoder framework.

The optimization is structured into three nested levels:

  1. Level 1 (Loss Search): Determines the optimal weights for various loss components.
  2. Level 2 (Architecture Search): Identifies the best arrangement of neural operations.
  3. Level 3 (Parameter Optimization): Trains the actual weights of the discovered network.

To manage the computational complexity of this triple-level problem, the authors employ a Hierarchical Bilevel Solver. A key component is the use of a separate image classifier to distinguish between "clear" and "degraded" images; the classification loss provides a high-level validation signal to guide the loss search toward more realistic results.

Experimental Results

The framework was evaluated on three distinct tasks: Low-Light Image Enhancement (LLIE), Single Image Haze Removal (SIHR), and Underwater Image Enhancement (UIE).

Visual examples of the three restoration tasks addressed: (a) Low-Light Image Enhancement, (b) Single Image Haze Removal, and (c) Underwater Image Enhancement.

Key results include:

  • Low-Light (LOL Dataset): Achieved a PSNR of 24.16, significantly outperforming the previous state-of-the-art (KinD at 20.25).
  • Dehazing (SIHR): Reached a PSNR of 27.05, surpassing the PSD model (25.19).
  • Underwater (UIEB): Achieved a PSNR of 23.00, compared to GLN (21.07).

Discussion and Insights

The research demonstrates that "physics-aware" NAS is superior to generic search for complex restoration tasks. By including Hadamard-based units in the search space, the resulting models better capture inherent image structures. Additionally, the search revealed that different tasks require different loss priorities; for instance, low-light enhancement benefits from perceptual losses, while underwater enhancement relies more on structural similarity (SSIM).

Conclusion

The Hierarchical Bilevel Learning framework offers a robust, automated solution for image restoration by integrating physical domain knowledge into the search process. While the search phase introduces initial computational costs and the current HNU is tailored for multiplicative degradation, the method sets a new benchmark for performance and visual fidelity in challenging environments. Future work aims to extend this approach to additive noise and high-level vision tasks.

Summary updated

BibTeX
@inproceedings{zhu2022hierarchical,
  author = {Guijing Zhu and Long Ma and Xin Fan and Risheng Liu},
  title = {{Hierarchical Bilevel Learning with Architecture and Loss Search for Hadamard-based Image Restoration}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2022},
  doi = {10.24963/ijcai.2022/245},
}
Figure