Trust That Tracks Where a Robot's Synthetic Training Ran Thin

Robots are increasingly trained mostly or entirely in simulation. A mobile manipulator that learned object handovers from 100,000 simulated scenes reached 80% success on real hardware in simple settings but 63.3% in complex ones (MobileH2R). A robot like this does not fail uniformly. It fails where its synthetic training was sparse or unrealistic. The person working beside it, however, sees only that it dropped something. Existing planners that reason about human trust summarize that person's trust as one number. When one number takes the hit, a failure on a poorly simulated wine glass lowers the person's willingness to let the robot handle the tin cans it does well. It can also leave that willingness too high on the next fragile, poorly simulated object.

We propose that a robot trained on synthetic data should model the human's trust as a vector. The vector holds one competence belief for each group of situations defined by how well the synthetic training covered that group, plus one belief about the robot's intent or risk-taking. The robot plans against this vector, using its own measured simulation-to-reality success rates per group as the ground truth it wants the human's beliefs to match. The claim is concrete and testable. How a person's trust spreads across situations after a failure can be captured by a fitted similarity pattern between situation groups. A planner that exploits that pattern should cut both needless human interventions and dangerous over-trust on high-risk, poorly covered objects, compared with a single-number trust planner. It should also beat simply telling the person where the robot is weak.

Where this sits

The planning machinery comes from Planning with Trust for Human-Robot Collaboration. That paper casts trust as a hidden variable in a partially observable Markov decision process (a planner that keeps a probability distribution over things it cannot see, here the human's trust). It models human intervention as a softmax over risk and trust, and it cut unnecessary interventions by up to 54% in simulation on a table-clearing task with low-, medium-, and high-risk objects. It also assumes a single trust scalar and a fixed robot capability. Those are exactly the two assumptions that break for a simulation-trained robot, whose capability varies by situation in a way that is unknown before deployment.

The motivation for splitting trust comes from Robot Capability and Intention in Trust-based Decisions across Tasks. In a 400-person online drone delegation game, that paper showed that capability trust transfers between tasks that share skill demands, and that a model separating capability, intention, and a residual fit delegation better (lower AIC/BIC) than one-number models. That study did not plan with the decomposition. It also left open whether the pattern holds with physical stakes, and it indexed capability by task rather than by any property of how the robot was built. A Trust Model for Supply Chain Management makes the same competence/integrity split for contracting agents. It supports the principle that honest capability failures must be separated from deliberate misbehavior, but it offers nothing specific to robots.

The synthetic-data literature constrains the design. On the Equivalency, Substitutability, and Flexibility of Synthetic Data finds that the synthetic-to-real "exchange rate" varies with scene density and motion dynamics, and that behavioral complexity in the simulation matters more than visual realism. This implies the robot's real competence should be broken into groups by coverage of behaviors and conditions, not by object appearance alone. AUGCAL shows two things we build on:

  • Models adapted from synthetic to real data become overconfident on their mistakes.
  • Post-hoc temperature scaling can worsen calibration under large shift.

We therefore estimate the robot's per-group competence from held-out real trials rather than from its raw softmax confidence. Finally, the closest work that joins trust and synthetic data, TruPercept, uses a game-engine world as a testbed for a single-scalar trust score over peer vehicles. Auditing and Generating Synthetic Data with Controllable Trust Trade-offs scores datasets on five technical "trust" axes. Neither models a human's changing beliefs about an agent whose competence was shaped by synthetic training, which is the object of this proposal.

Core Claim

Let the situations the robot faces be partitioned into $K$ coverage groups. Each group is defined before deployment by how densely and how realistically it was represented in simulation, for example well-simulated rigid cans versus rarely simulated thin-walled glasses in clutter. The human's hidden trust state is $\theta = (c_1, \dots, c_K, \iota)$. Here $c_k$ is the person's belief that the robot succeeds in group $k$, and $\iota$ is their belief that the robot's risk-taking is aligned with theirs. After the robot succeeds or fails on an object in group $j$, the competence beliefs update as

$$c_k \leftarrow c_k + \eta \, w_{jk} \, (y - c_k),$$

where $y \in \{0,1\}$ is the observed outcome, $\eta$ is a learning rate, and $w_{jk} \in [0,1]$ is how strongly evidence about group $j$ spreads to group $k$ in the person's mind. The intervention model is the softmax of Chen et al., with the relevant $c_k$ and $\iota$ in place of the scalar.

The single-number model is the special case where every $w_{jk} = 1$. Fully independent groups is the identity matrix. The empirical question is where real people fall between these two cases, and whether their spread follows the robot's actual failure structure or superficial similarity. The planning claim follows from this. Given a fitted $w$ and the robot's measured per-group real success rates $r_k$, the planner chooses actions to push each $c_k$ toward $r_k$. These actions include which object to attempt next, when to hand an item back, and when to state a limitation. The main danger it guards against is high $c_k$ with low $r_k$ on high-risk groups.

Research Program

The work has three parts.

Build a robot with deliberately uneven synthetic coverage. We train a grasp-and-place policy for the table-clearing task entirely in simulation. We control how many scenes each object family and clutter condition receives, and some families have physics that is known to be hard to simulate (fragile, transparent, deformable). We then measure real success per coverage group on held-out trials to obtain $r_k$. We also train a variant fine-tuned on a small fraction of real data, following the substitutability recipe of Chang et al. This tests whether per-group unevenness survives the standard fix. For the large online study, robot outcomes in videos or rendered episodes are sampled from the measured $r_k$. This keeps human exposure controlled while staying faithful to how the real robot actually fails.

Measure how human trust spreads. In an online study patterned on Chen et al.'s table-clearing protocol and Xie et al.'s delegation game, participants supervise the robot across mixed sequences of objects and may intervene. Failure placement is randomized across coverage groups. We fit $w$ by maximum likelihood on intervention choices and compare held-out prediction against the scalar, independent, and task-indexed capability/intention models.

Plan against the fitted model. We extend the Trust-POMDP state to $\theta$ and solve it with the same point-based solver. We compare policies in a new online study, followed by a small physical study that addresses Xie et al.'s open question about embodied stakes.

Baselines

The naive reference is a myopic robot that always attempts the next object with its best grasp and never reasons about the human. This is the baseline Chen et al. compare against.

The strongest existing method is the scalar Trust-POMDP of Planning with Trust for Human-Robot Collaboration. The reported result is a reduction of unnecessary interventions of up to 54% relative to the myopic baseline in simulation, with significantly higher cumulative reward ($p<0.01$). We will run it on the identical task, object sequences, participant pool, and solver budget. It will be refit on the same training interaction data we use to fit $w$, so that both models are estimated from the same data. Beating it means two things together:

  • a significant reduction in over-trust events, meaning no intervention on a high-risk object the robot then fails;
  • no increase in unnecessary interventions, and higher cumulative reward under Chen et al.'s reward.

The strongest non-trust competitor is a calibrated-self-confidence robot. It estimates its own per-group success, using held-out real trials rather than temperature scaling given AUGCAL's evidence, and asks the human for help whenever that estimate falls below a threshold. The threshold is tuned on the same data. It must be beaten on the same metrics.

Three ablations isolate the key ingredient. A vector model whose groups are formed at random, with the same $K$, tests whether coverage-aligned grouping matters or just extra parameters. A vector model grouped by object appearance only tests whether coverage is the right axis. A disclosure-only condition has the scalar planner plus a verbal statement of which groups were rarely simulated.

Why not just…?

Why not just keep a single trust number, which is cheaper and plugs into existing solvers? Xie et al. already found decomposed models fit delegation better, but in a task-indexed, non-embodied game. If our fitted $w$ is close to all ones, the scalar model is the right answer for this setting, and we will report that. The held-out likelihood comparison and the fitted $w$ settle this directly.

Why not just have the robot know its own weaknesses and ask for help? This is the most serious threat. A well-calibrated robot that defers in low-$r_k$ groups might prevent most dangerous over-trust with no human model at all. Its expected weakness is the other side of the problem. It cannot repair a person's collapsed trust in well-covered groups after a salient failure, so needless interventions there should persist. If the calibrated-deferral baseline matches our planner on both over-trust and unnecessary interventions, the planning contribution fails, and the result becomes a behavioral finding about how trust spreads.

Why not just tell the person where the synthetic training was thin? Disclosure is cheap and may by itself reshape $w$. We test it as a separate arm and in combination with the planner. If disclosure alone reaches the planner's numbers, we would recommend disclosure. Xie et al.'s gap between what participants reported about trust and what they actually delegated suggests stated information may not change behavior, but that has to be measured, not assumed.

Why not just fine-tune on some real data and remove the gap? Chang et al. show that 20–40% real data can match or exceed full-real training on average. Average parity can still hide uneven per-group failure. Our fine-tuned variant measures whether the $r_k$ remain spread out. If they collapse to near-uniform values, the problem this proposal targets largely disappears for that robot, and we will report that.

Decisive Checks

The first check is behavioral. On held-out participants, the fitted spread matrix should give significantly better intervention prediction than the scalar model. Its off-diagonal structure should correlate more with coverage groups than with appearance groups. If both fail, the proposal stops at a negative result on how trust spreads.

The second check is the planning comparison against the scalar Trust-POMDP, calibrated deferral, and disclosure-only arms, on over-trust events for high-risk, low-coverage objects and on unnecessary interventions for high-coverage objects. The random-grouping ablation must lose to coverage grouping for the gain to be attributed to synthetic coverage.

The third check is the physical study with a smaller sample. The same ordering of conditions should hold. Its absence would indicate that the online effect does not survive real stakes.