Human Feedback Attack on Online RLHF: Attack and Robust Defense

Chenye Yang, Mo Lyu, Guanlin Liu, Lifeng LaiIEEE TSP 2025 · OpenReview

Abstract

In this paper, we investigate human feedback attacks on online Reinforcement Learning from Human Feedback (RLHF) algorithms. The attacker’s goal is to force the victim RLHF algorithm to eventually learn a suboptimal policy while inducing a small attack cost. We propose an adversarial attack strategy, and prove that it is successful in terms of misleading the online RLHF algorithm to learn the suboptimal target policy. We also propose a robust defense online RLHF algorithm. We show that the proposed algorithm is robust to any attacker whose attack cost is bounded by a budget. The simulation results validate our theoretical analysis.
BibTeX
@article{yang2025human,
  author = {Chenye Yang and Mo Lyu and Guanlin Liu and Lifeng Lai},
  title = {{Human Feedback Attack on Online RLHF: Attack and Robust Defense}},
  journal = {IEEE Transactions on Signal Processing},
  year = {2025},
  url = {https://openreview.net/forum?id=ZFivhxibZ0},
}

Loading related content...

Figure