In the era of "Model-as-a-Service," the most powerful AI models are often locked behind APIs. Researchers and developers can send inputs and receive outputs, but they cannot access the model's internal weights or gradients. This "black-box" constraint makes traditional fine-tuning impossible. To adapt these models to specific tasks, we must instead optimize the prompt—the sequence of tokens or continuous embeddings that guide the model's behavior.
The fundamental challenge in this field is the "query tax." Standard optimization techniques like derivative-free optimization (DFO) or reinforcement learning typically require thousands, or even hundreds of thousands, of API calls to converge. When each call costs money and incurs latency, these methods become prohibitively expensive. Recent research focuses on slashing this query volume while maintaining high performance.
The Low-Dimensionality Shortcut
Why is it possible to optimize a prompt for a model with billions of parameters using only a few hundred queries? The answer lies in intrinsic dimensionality. Tianxiang Sun and colleagues demonstrated in Black-Box Tuning for Language-Model-as-a-Service that pre-trained models have a very low intrinsic dimension.
Instead of searching for an optimal prompt in a high-dimensional space (where every token or embedding dimension is a variable), we can project the problem into a tiny, randomly generated subspace. By performing derivative-free optimization within this subspace, we can find effective prompts with orders of magnitude fewer queries. This concept has been extended to the visual domain by Zhaogeng Liu in Query Efficient Black-Box Visual Prompting with Subspace Learning, which slashes the query requirements for vision models by over 95%.
Prompting as Reinforcement Learning
If we treat the black-box model as an environment, prompt optimization becomes a Reinforcement Learning (RL) problem. The "agent" generates a prompt, and the model's performance on a task provides the "reward."
To make this query-efficient, researchers use variance-reduction and actor-critic frameworks. Shizhe Diao introduced Black-box Discrete Prompt Learning (BDPL) (paper), which uses policy gradients to navigate the discrete space of natural language tokens. Similarly, Fares Fourati proposed ACING (paper), an actor-critic approach that discovers instructions outperforming human-written ones in just a handful of iterations. The intuition here is that we don't need to see the gradients if we can learn a policy that predicts which prompt modifications are likely to yield higher rewards.
From Exhaustive Search to Structured Feedback
A newer frontier moves away from pure mathematical optimization toward agentic refinement. Instead of treating the prompt as a vector of numbers, these methods treat it as a structured program or a set of instructions that can be critiqued.
- Self-Evolving Loops: Eshaan Agarwal developed PromptWizard (paper), which uses a "self-evolving" feedback loop. The system looks at where the model failed, critiques the current prompt, and suggests improvements. This structured feedback is far more sample-efficient than random search.
- Semantic Memory: Priyanshu Gupta and the team behind MetaReflection (paper) take this further by generalizing past failures into a "semantic memory." By learning from mistakes across different tasks, the system can refine prompts with significantly fewer live model calls.
- Prompt Ensembling: Rather than searching for one "perfect" prompt, Bairu Hou introduced PromptBoosting (paper). This method uses AdaBoost to ensemble multiple sub-optimal prompts. By focusing on the "hard" examples that previous prompts missed, it achieves high accuracy in as few as ten forward passes.
Hybrid and Collaborative Adaptation
When the constraints are slightly relaxed—for instance, if we have a small local model alongside the black-box API—we can use hybrid strategies.
Zhengbo Wang proposed CraFT (paper), which splits the work: it uses derivative-free optimization to tune the input prompt for the black-box model, but uses standard gradient-based learning to refine a small "adapter" at the output. This collaborative approach provides the best of both worlds: the power of the massive black-box model and the efficiency of gradient descent.
Finally, for scenarios where data privacy is paramount, Zihao Lin introduced Fed-BBPT (paper), a federated learning framework. It allows multiple users to collaboratively optimize a shared prompt generator without ever sharing their private data or needing access to the central model's weights.
Summary of the Direction
The shift in black-box adaptation is clear: we are moving from brute-force search to intelligent refinement. By leveraging the low intrinsic dimensionality of models, employing RL with high sample efficiency, and using agentic feedback loops, researchers are making it possible to "fine-tune" the world's largest AI models on a shoestring budget.