Reinforcement Learning-Based Sequential Parameter Tuning for Image Signal Processing

Xinyu Sun, Bing Li, Congyan Lang, Zhikun Zhao, Juan Wang, Weihua Xiong, Weiming Hu, Long ChengIEEE TPAMI 2025 · DOI

Abstract

Hardware image signal processing (ISP) transforms RAW inputs into high-quality RGB images through a series of processing modules, each with numerous tunable parameters. Traditionally, these parameters are manually tuned by imaging experts, a time-consuming and subjective process. Recent deep learning approaches predict ISP parameters, but often treat the process as a black box and overlook the intrinsic relationships among ISP modules. To address these fundamental issues, we introduce a novel ISP parameter optimization model based on single-agent reinforcement learning (RL) (i.e., SARL-ISP), formulating the hardware ISP parameter tuning as a sequential optimization problem. During the optimization process, the agent updates ISP parameter tuning strategies for different tasks through interaction with the environment. In order to explore the influence of the sequential structure of hardware ISP modules and the coupling relationships among ISP parameters on the tuning process, we further propose a sequential ISP framework based on collaborative multi-agent RL (i.e., MARL-ISP). Specifically, the serialized parameter tuning module (SPTM) realistically simulates the process of manual prediction and module pipeline. Additionally, the feature selection module (FSM) facilitates the transmission and fusion of agent features, thereby selecting more appropriate feature inputs for downstream tasks. Extensive experiments across various tasks (e.g., object detection, instance segmentation) validate the effectiveness and efficiency of our models. Even with minimal training data, our models also outperform current state-of-the-art methods in both quantitative metrics and qualitative evaluations.
BibTeX
@article{sun2025reinforcement,
  author = {Xinyu Sun and Bing Li and Congyan Lang and Zhikun Zhao and Juan Wang and Weihua Xiong and Weiming Hu and Long Cheng},
  title = {{Reinforcement Learning-Based Sequential Parameter Tuning for Image Signal Processing}},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2025},
  doi = {10.1109/tpami.2025.3642837},
}

Loading related content...

Figure