Mastering Visual Reinforcement Learning via Positive Unlabeled Policy-Guided Contrast

Zehua Zang, Qirui Ji, Rui Wang, Kai Li, Lixiang Liu, Fuchun Sun

Abstract

Reinforcement learning (RL) has garnered considerable attention in recent years. A fundamental yet challenging problem in this paradigm is the effective perception of high-dimensional environmental information, leading to the emergence of Visual Reinforcement Learning (VRL). This sub-field focuses on learning representations from pixel observations to optimize decision-making policies. In this article, we provide a comprehensive analysis of existing benchmark frameworks and highlight a persistent paradox that challenges current approaches: during different training phases, the exploration of visual semantic information can both enhance and hinder the quality of learned representations. Furthermore, we reveal that the issue of over-redundancy commonly limits the sample efficiency of baseline methods. To address these limitations, we propose a novel plug-and-play approach for VRL. Our method employs a positive unlabeled policy-guided contrastive learning framework to jointly capture anti-redundant and policy-relevant pixel semantics during training. We validate the effectiveness of our approach through extensive benchmarking experiments, demonstrating its superior performance over existing methods in pixel environments.
BibTeX
@inproceedings{zang2025mastering,
  author = {Zehua Zang and Qirui Ji and Rui Wang and Kai Li and Lixiang Liu and Fuchun Sun},
  title = {{Mastering Visual Reinforcement Learning via Positive Unlabeled Policy-Guided Contrast}},
  booktitle = {ICIC},
  year = {2025},
  url = {https://openreview.net/forum?id=sucSzR3plo},
}

Loading related content...

Figure