GOOD: Decoding-Time Black-Box LLM Alignment
Its authors withdrew this paper, so there is no full text to read or summarize.
Lists
Only you see this
Abstract
Large Language Models (LLMs) have demonstrated immense potential across various applications. However, aligning these models with specific real-world tasks and human preferences typically requires resource-intensive fine-tuning processes such as Supervised Fine-Tuning (SFT) and Reinforcement Learning from Human Feedback (RLHF). In this paper, we propose GOOD (Guided Online Optimal Decoding), a novel alignment method that enhances pre-trained models without the need for parameter fine-tuning. We observed that the alignment-related behavior of one model can be used to guide another model, and based on this insight, we proposed the GOOD method. Utilizing a pair of guiding models, GOOD identifies critical positions related to alignment and adjusts the model’s output dynamically during the response generation. Notably, the interaction between the guiding models and the guided model occurs at the string level, enabling GOOD to be applied to align even black-box models. Experiments show that GOOD can achieve performance comparable to or even surpassing direct fine-tuning in terms of comprehensive capability and harmless generation, reaching relative scores of 108\% and 105\% respectively. Even in weak-to-strong alignment, it can recover up to 94\% of the performance of directly fine-tuned models. GOOD can also be applied to enhance already aligned models (improving pass@1 by 52\% in code enhancement), making it compatible with various existing alignment techniques.
BibTeX
@misc{fang2025good,
author = {Hainan Fang and Di Huang and Yuanbo Wen and Yunpu Zhao and Tonghui He and QiCheng Wang and Shuo Wang and Rui Zhang and Qi Guo},
title = {{GOOD: Decoding-Time Black-Box LLM Alignment}},
year = {2025},
url = {https://openreview.net/forum?id=cayKVPCrOP},
howpublished = {OpenReview},
note = {Submitted to ICLR 2025},
}Loading related content...