Conversational recommenders that serve brand-new users (the "cold-start" case, where the system knows nothing about the person yet) are almost always developed against user simulators, meaning language models prompted to play the customer. Recent human studies show these simulators are systematically too easy. They reveal their preferences up front, quietly go along with the agent when it makes a mistake, and report inflated satisfaction. On τ-bench, simulator-judged agent success reached up to 77.8% while real humans achieved 63.6% (Mind the Sim2Real Gap in User Simulation for Agentic Tasks). Cold-start elicitation suffers most from this bias. A simulator that volunteers everything makes asking questions look unnecessary, so policies tuned on it under-ask real people.
We propose to treat the simulator as a cheap, biased instrument and to correct it with a small, deliberately chosen set of real human interactions. The correction is a sparse Gaussian process, a Bayesian regression model that summarizes a function through a small set of anchor locations called inducing points. It models the difference between how a real user and the simulator respond in a given dialogue state. The twist is that the inducing points are not free parameters tuned for computational convenience. They are the dialogue states where we will pay to observe a human. Choosing them is therefore an experimental-design decision: which states to show to people so that the corrected model best predicts how humans will judge the policies we actually intend to deploy.
Core Claim
The simulator's error is not a uniform offset. It is concentrated in identifiable regions of dialogue-state space: after agent mistakes, in the first few cold-start turns, and wherever preferences are only partly revealed. If that is true, a sparse discrepancy model whose inducing points are placed in those regions will predict human outcomes from a fixed human budget $m$ more accurately than three alternatives: global recalibration, simulator fine-tuning on the same $m$ transcripts, or randomly collected human data.
Concretely, let $x$ be a dialogue state, meaning an embedding of the agent's last action together with a summary of the user's revealed information, the turn index, and whether the agent just erred. Let $y$ be the user's next-turn outcome, such as satisfaction, frustration, or abandonment. We model
$$y_{\text{human}}(x) = \rho\, y_{\text{sim}}(x) + \delta(x) + \epsilon,$$
where $y_{\text{sim}}(x)$ is the simulator's response, averaged over its rollouts, $\rho$ is a learned scale, $\delta$ is a Gaussian process discrepancy, and $\epsilon$ is noise. The discrepancy $\delta$ is summarized by inducing values at locations $Z$, and every element of $Z$ is a state at which a human label is collected. The claim fails if a scalar $\rho$ plus a constant offset matches the full model. That outcome would mean the gap is not localized, and it is the first thing the experiments test.
Relation to Prior Work
Three strands supply the method. Large scale variable fidelity surrogate modeling couples cheap low-fidelity and scarce high-fidelity data with inducing (Nyström base-point) approximations. Its purpose is speed on physical simulations; the inducing points reduce computation and are subsampled by self-covariance. We borrow the two-fidelity structure but make the inducing points the human-labeling budget itself. Inducing Point Allocation for Sparse Gaussian Processes in High-Throughput Bayesian Optimisation places inducing points with a determinantal point process, a sampler that trades off per-point quality against diversity, and "gerrymanders" them toward promising optima. We reuse that decomposition with a different quality term: how often candidate policies visit a state, weighted by current uncertainty about $\delta$ there. The diversity term addresses the tension the file identifies between concentrating capacity and covering the heterogeneous tastes of cold-start users. Probabilistic selection of inducing points in sparse Gaussian processes and Adjusting Model Size in Continual Gaussian Processes: How Big is Big Enough? show that the number of inducing points can adapt to data complexity. We use the second paper's bound-gap criterion as a stopping rule for human collection, answering how many humans are enough. The variational machinery is standard (Scalable Variational Gaussian Process Classification).
Two papers motivate the problem. Mind the Sim2Real Gap in User Simulation for Agentic Tasks documents front-loading and quiet pivoting after errors, which are exactly the state-dependent failures our hypothesis predicts. ConvApparel shows that human-likeness scores are near zero for all simulators, and that fine-tuned and in-context simulators generalize better to an unseen "bad" recommender. That result is the strongest evidence against us, because it suggests simply fine-tuning the simulator.
One paper constrains the design. Gaussian Processes for Fast Policy Optimisation of POMDP-based Dialogue Managers used GP uncertainty to learn dialogue policies within about 400 dialogues, but it did so over hand-built summary spaces and only in simulation. Its lesson is that GP kernels over dialogue states are workable when the state is summarized well. Our proposal differs in that we never replace the simulator. We model only its residual, and we evaluate against humans. Cold-start recommenders such as Task-adaptive Neural Process for User Cold-Start Recommendation and MAMO are offline rating predictors. We do not compete with them. They define the downstream regime of few interactions and heterogeneous users in which elicitation policies are judged.
Research Program
We will build the discrepancy model over frozen text-encoder embeddings of dialogue states, with an additive kernel over the explicit features (turn index, fraction of the preference profile revealed, agent-error flag). This keeps the model interpretable: the learned lengthscales show where the gap lives.
Selection proceeds in rounds. We roll out candidate recommender policies against the simulator to estimate visitation over states. We then score pool states by visitation times the posterior variance of $\delta$, sample a diverse batch from the resulting determinantal point process, reveal the human outcomes, and refit. Collection stops when the bound gap from the continual-GP work falls below a threshold.
The corrected model then serves two uses. First, it predicts the human-judged quality of a new agent before deployment. Second, it acts as a reward for choosing among cold-start elicitation policies, such as how many clarifying questions to ask before the first recommendation.
We will also analyze the learned $\delta$. We will report where the mass of the discrepancy falls, and whether that location is stable across simulator families. The latter matters because Zhou et al. find that fidelity does not track general model capability.
Proposed Experiments
The main testbed is ConvApparel, in a pool-based protocol. The 4,146 human-AI shopping conversations, with their first-person turn annotations, form the pool of states. "Collecting a human" means revealing a pool state's human label. Simulator responses are generated at the same states by zero-shot, in-context, and fine-tuned LLM simulators. Held-out targets are human outcomes on held-out conversations, including the counterfactual "bad" recommender, which tests whether the correction transfers to an agent whose error states were rare during selection.
Budgets are $m \in \{25, 50, 100, 200\}$ human-labeled states. The metrics are negative log-likelihood and calibration of predicted per-turn human satisfaction and frustration, absolute error in predicted agent-level satisfaction, and Spearman correlation between predicted and human rankings of candidate agents.
The second testbed is τ-bench, using the 495 human traces. Here the target is agent success. Sim-only prediction starts at the documented 77.8% ceiling against the human 63.6% (Mind the Sim2Real Gap in User Simulation for Agentic Tasks), and the question is how much of that 14-point gap each method closes per human trace.
For the downstream cold-start check, we select an elicitation policy on ConvApparel using each corrected reward. We then compare the chosen policy's held-out human satisfaction against that of the policy the raw simulator would pick.
Baselines
The naive reference point is simulator-only prediction: the simulator's own outcome, used as if it were human. On τ-bench this is the reported 77.8% simulated success against 63.6% human (Mind the Sim2Real Gap in User Simulation for Agentic Tasks). Any method must close a clear share of that gap to matter.
The strongest existing method is a simulator fine-tuned, or given in-context examples, on the same $m$ human conversations, following ConvApparel. It is the strongest competitor because it is what practitioners already do and because it generalizes counterfactually better than prompting. For fairness it receives exactly the same human data (the transcripts containing our selected states, or $m$ randomly chosen ones, whichever helps it more) and a matched tuning budget over learning rate and epochs. No reported number was found on Lacuna for fine-tuned-simulator prediction error on ConvApparel outcomes, so we will measure it ourselves. Beating it means lower held-out error and better agent ranking at every $m \le 100$, with the advantage largest on the "bad" recommender.
A global recalibration baseline fits only $\rho$ and a constant offset from the same $m$ human labels. It directly tests the localization claim.
A human-only Gaussian process uses the same $m$ labels and no simulator. It tests whether the simulator adds anything at all.
Selection ablations keep the discrepancy model fixed and vary only how the $m$ states are chosen: uniformly at random, by k-means centroids in embedding space as in the geometric heuristics, by variance alone without visitation weighting, and by visitation alone without the diversity term. These isolate the key ingredient, namely choosing human queries as policy-relevant, diverse inducing points. The proposal counts as confirmed only if the full criterion beats random selection by a margin larger than seed variance at small $m$.
Why not just…?
Why not just fine-tune the simulator on the human data? This is the real threat. The evidence against it is only suggestive. Fine-tuned simulators in ConvApparel still score near zero on human-likeness, and helpfulness training suppresses the flaws that matter (Mind the Sim2Real Gap in User Simulation for Agentic Tasks). Fine-tuning also gives no uncertainty and no guidance on which humans to collect. The head-to-head comparison at equal $m$ settles the question. We also run the hybrid, a fine-tuned simulator plus our discrepancy model. If the hybrid does not beat fine-tuning alone, the discrepancy model adds nothing.
Why not just apply a global correction, shifting simulator scores down by a learned constant? If the gap were uniform, that would be sufficient and much simpler. Zhou et al.'s observation that simulators diverge specifically after agent errors argues otherwise. The recalibration baseline together with the learned lengthscales on the error-flag and turn-index features will show directly whether the gap is localized. If recalibration ties the full model, we will report that the gap is effectively global and that the selection machinery is unnecessary.
Why not just collect the $m$ humans at random, or skip the simulator? Random collection is the natural default, and with large $m$ it should catch up. The human-only Gaussian process and the random-selection ablation measure exactly how many human interactions the simulator and the targeted selection save. We will report that saving as the break-even budget $m$ at which each baseline matches our method.
Decisive Checks
Three results would validate the idea. First, at $m = 50$ on ConvApparel, the full model must beat both recalibration and fine-tuned simulators in held-out calibration and agent ranking, including on the unseen bad recommender. Second, the learned discrepancy must concentrate in post-error and early cold-start states, consistently across simulator families. Third, the elicitation policy chosen with the corrected reward must ask more clarifying questions than the simulator-chosen policy and earn higher held-out human satisfaction.
Any one of three results would falsify it: recalibration ties the full model, random selection ties targeted selection, or the gain vanishes on the counterfactual agent.
References
- Mind the Sim2Real Gap in User Simulation for Agentic Tasks
- Large scale variable fidelity surrogate modeling
- Inducing Point Allocation for Sparse Gaussian Processes in High-Throughput Bayesian Optimisation
- Probabilistic selection of inducing points in sparse Gaussian processes
- Adjusting Model Size in Continual Gaussian Processes: How Big is Big Enough?
- Scalable Variational Gaussian Process Classification
- ConvApparel: A Benchmark Dataset and Validation Framework for User Simulators in Conversational Recommenders
- Gaussian Processes for Fast Policy Optimisation of POMDP-based Dialogue Managers
- Task-adaptive Neural Process for User Cold-Start Recommendation
- MAMO: Memory-Augmented Meta-Optimization for Cold-start Recommendation