Illuminating Emotion–Action Consistency in Negotiating Agents with MAP-Elites

When a negotiating agent learns by chasing reward, the easiest wins come early. Smiling while conceding, or flattering while exploiting, pays off in the first few rounds. Human studies show that trust later collapses when an agent's emotional signals do not match what it actually does. That makes social interaction a deceptive search problem: the strategies that look best early lead away from the strategies that are best over many rounds. Optimizers that greedily copy the current winner tend to lock onto these early wins, a failure known as premature convergence. This proposal tests whether MAP-Elites can avoid that trap. MAP-Elites is an evolutionary method that keeps the best solution found in each cell of a grid of behaviors, instead of keeping only the single best solution overall. Here the grid describes how cooperatively an agent acts against how warm its emotional signals are. The agents being evolved have numeric value weights, which decide what they offer, and a fine-grained emotion-expression map, which decides how they sound while offering it.

The core bet, in two sentences, is this. If the behavior grid places emotion and action on separate axes, the archive is forced to keep "consistent but unpromising" agents, such as an openly angry hard bargainer, that greedy search throws away. Those agents are the stepping stones from which long-horizon, high-trust strategies evolve. A second bet concerns emotional detail. The detail matters in what the agent can express, which is the genome, and not in the grid used to organize the search. This resolves the tension between richer emotion taxonomies and MAP-Elites' exponential growth in grid size.

What exists and what does not

Three strands of prior work motivate the project, and they have not been combined. On the agent side, Towards Humanoid builds a large language model (LLM) agent from three parts: an internal module of values and needs, a pursue module that turns values into goals, and a desire module that overrides goals when needs are urgent. The authors name fixed, hand-written value profiles as their main limitation. We keep their decomposition but replace the fixed profile with an evolved numeric weight vector. The Influence of Emotion Expression on Perceptions of Trustworthiness in Negotiation supplies the phenomenon we exploit. In their games, people preferred cooperative agents that smiled, and they preferred selfish agents that looked angry over selfish agents that smiled (+25% in each case), because consistent signals make an agent predictable. That study uses scripted, categorical displays and no learning. DialogXpert learns emotion-aware dialogue policies with a Q-network on top of an LLM's proposed actions. It is a single optimizer with a coarse emotion classifier, which is exactly the setting where convergence onto one degenerate conversational mode is expected. An Explainable Emotion Alignment Framework couples a pleasure-arousal-dominance emotion state to decisions, but it is limited to 7 emotion types and does no population search.

On the search side, PBT-MAP-Elites evolves policy weights together with learning hyperparameters inside a behavior grid, and it was the first method to solve the deceptive Humanoid-Trap task within 150M timesteps. Its behavior descriptors are purely physical, such as foot contacts and positions. We borrow its insight that the parameters controlling the learner should evolve alongside the policy, and we apply it to value weights. Multiple-Frequencies Population-Based Training shows that population-based training (PBT), which repeatedly replaces weak agents with mutated copies of strong ones, can collapse: it scored IQM 16,171 on Brax Humanoid at 1B steps, below random search at 17,713. Their fix, MF-PBT, reaches 23,793. Understanding the Importance of Evolutionary Search in Automated Heuristic Design finds the same greedy trap when LLMs search over code. Qian et al. prove that quality-diversity search can beat objective-driven search on deceptive problems by preserving stepping stones. These papers supply the method and the hypothesis. None of them touches social or affective agents.

Two results constrain the design. MAP-Elites grids grow as the product of the resolutions of all their axes, so a 40-dimensional emotion grid such as one built on the EmoNet-Face taxonomy cannot be gridded directly. The EmoNet-Face authors also report high expert disagreement on fine distinctions such as shame versus embarrassment. Both results push fine granularity out of the behavior grid and into the expression layer, where the fitness signal can decide whether the detail helps.

Core Claim

In repeated negotiation, where trust builds or erodes across rounds, a MAP-Elites archive whose two behavior axes are action cooperativeness and expressed warmth will find agents with higher long-horizon payoff and partner trust than greedy search, PBT, MF-PBT, or random search, all given the same evaluation budget. We predict that the best final elites will trace their ancestry through low-fitness cells in which signal and action were consistent. A version of the archive whose grid covers action alone should lose most of this advantage, because it cannot tell a "smiling hypocrite" apart from an "honest hard bargainer."

Research Program

The agent keeps the three-module structure of Towards Humanoid. Its genome has two parts. The first is a value-weight vector covering self-enhancement, benevolence, fairness concern, and a resource "need" with a decay rate. The pursue module uses these weights to set the agent's reservation price and its offer schedule. The second part is an expression map: a small parametric function that takes an appraisal of each round (gain or loss relative to expectation, and the partner's perceived fairness) and outputs an intensity vector over the 40 EmoNet-Face categories. A fixed, small LLM turns the offer plus the top-weighted emotions into an utterance. The LLM itself is never trained. Search operates only on the numeric genome, which keeps per-evaluation cost far below the multi-day LLM search costs reported by Zhang et al..

The behavior descriptor has two axes, each split into 10 bins. The first is cooperativeness, measured as the mean share of the surplus the agent offers. The second is warmth, measured as the projection of the expressed emotion vectors onto a fixed axis. That axis is defined once, before search, as the valence direction obtained from the category ratings in the taxonomy, and it is frozen so that the grid cannot drift. Fitness is the agent's cumulative payoff over 20 rounds against a held-out population of partners.

The partners make the task deceptive. Their trust level rises when signal and action agree and falls sharply when they disagree, following the direction of the effect in Antos et al.. Exploitation pays well in rounds 1–3 and then dries up. We vary the strength of this deception with a single parameter, and at a setting of zero the task is not deceptive at all.

The language phase moves the archive's elites to CraigslistBargain. There the opponents are LLMs simulating buyers, and their trust is read from their replies rather than computed by a formula. A small human study then measures partner preference and the price gap between our agents' deals and human–human deals.

Baselines

The naive reference is the hand-coded value-driven agent of Towards Humanoid, using its default value profile and a 7-category emotion layer. It shows what fixed values achieve with no search.

Random search samples genomes uniformly from the same space and spends the same number of evaluations. Despite its simplicity it is a serious competitor, because Multiple-Frequencies Population-Based Training found it beating standard PBT on Humanoid. Greedy (1+1) evolution and standard PBT stand in for the greedy-trap behavior documented by Zhang et al..

The strongest diversity-preserving competitor is MF-PBT, which reported IQM 23,793 against 16,171 for PBT on Brax Humanoid at 1B steps. That number comes from locomotion, and no result on Lacuna was found for MF-PBT on negotiation or trust games, so we will run it ourselves with matched population size. In language negotiation the strongest existing method is DialogXpert, which reports success above 94% with a Vicuna 13B prior and above 97% with a Qwen 2.5 14B prior on its proactive-dialogue suite, CraigslistBargain included. We will rerun it with the same small LLM we use, so that the comparison isolates the search method rather than the size of the prior. A multi-objective quality-versus-dissimilarity optimizer in the spirit of Gupta et al. tests whether a grid is needed at all.

Every method gets the same evaluation budget of 20,000 episodes, the same partner population, the same genome space, and the same number of tuning trials for its own hyperparameters (8).

Three ablations isolate the ingredients. The first uses a grid over action alone. The second uses a grid over emotion alone. The third keeps the joint grid but restricts the genome to a 7-category emotion map.

Beating the field means three things. MAP-Elites must reach higher median long-horizon fitness than MF-PBT and random search across 10 seeds, with non-overlapping bootstrap intervals, at deception settings at or above the midpoint. The advantage should disappear when deception is set to zero. On CraigslistBargain, the agents must match DialogXpert's deal rate while showing higher partner trust and a smaller gap from human deal prices.

Why not just…?

Why not just prompt an LLM with many diverse personas? Prompting yields variety in tone, but nothing searches for the persona that works best. Zhang et al. found that standalone LLM sampling plateaued even after 10,000 queries. Ma et al. report inconsistent "role illusion" behavior from generic LLM agents. We include a persona-prompt population of matched size, scored on the same fitness.

Why not just random search over value weights? This is the real threat. If the space is small, or the trap is shallow, random sampling may find good agents as easily as MAP-Elites does. The deception sweep tests this directly. If random search matches MAP-Elites at high deception, our stepping-stone claim is wrong. In that case we would report the archive as an analysis tool, not an optimizer.

Why not just MF-PBT, which already fixes PBT's collapse? MF-PBT protects diversity in hyperparameters but has no notion of behavior. It can still converge on hypocritical strategies, provided that it does so with robust hyperparameters. The head-to-head comparison, together with lineage analysis of what each method's population looks like at round 3 versus round 20, will settle this.

Why not just use coarse emotions, as DialogXpert does? The 7-category ablation answers this. If it matches the 40-category genome, emotional granularity does not matter for this task, and we will say so. The evidence from Antos et al. concerns consistency, not granularity, so this result is genuinely uncertain.

Decisive Checks

Four results decide the project. The first is the fitness-versus-deception curve: MAP-Elites should gain an advantage only once the deception parameter is above zero. The second is the lineage audit: the fraction of final top elites whose ancestry passes through below-median cells in which signal and action were consistent. If this fraction is low, the gains come from diverse sampling rather than from stepping stones. The third is the comparison of the joint grid against the action-only grid. The fourth is a transfer check that guards against circularity, since we built the trust rule into the simulated partners ourselves. Elites must keep their ranking when evaluated against LLM buyers on CraigslistBargain and in the human preference study. If the ranking holds only under the hand-built trust rule, the main claim fails.