Abstract
Great storytellers know how to take us on a journey. They direct characters to act -- not necessarily in the most rational way -- but rather in a way that leads to interesting situations, and ultimately creates an impactful experience for audience members looking on. If audience experience is what matters most, then can we help artists and animators directly craft such experiences, independent of the concrete character actions needed to evoke those experiences? In this paper, we offer a novel computational framework for such tools. Our key idea is to optimize animations with respect to simulated audience members' experiences. To simulate the audience, we borrow an established principle from cognitive science: that human social intuition can be modeled as "inverse planning," the task of inferring an agent's (hidden) goals from its (observed) actions. Building on this model, we treat storytelling as "inverse inverse planning," the task of choosing actions to manipulate an inverse planner's inferences. Our framework is grounded in literary theory, naturally capturing many storytelling elements from first principles. We give a series of examples to demonstrate this, with supporting evidence from human subject studies.
TL;DR
The authors propose Inverse Inverse Planning (I2P), a computational framework that optimizes animations to evoke specific inferences in an audience. By modeling the audience as a Bayesian "inverse planner," the system can automatically generate character movements that clearly convey complex narrative elements like intent, irony, and plot twists.
The Communicative Gap in Rational Planning
In traditional computer animation and AI, agents are often programmed to be "rational"—they take the shortest or most efficient path to a goal. However, real-world storytelling is rarely about efficiency. A character might take a circuitous route to show hesitation, or perform a redundant action to signal help.
The "gap" identified by the authors is that rational actions are often ambiguous to an observer. If a baby crawls toward a spot between a cupcake and a candy bar, a viewer can't be sure which one she wants. To a storyteller, the goal isn't just for the baby to reach the cupcake; it’s for the audience to know she wants the cupcake. This paper provides a mathematical way to automate that "expressive" choice.
Methods: Modeling the Audience
The framework relies on a recursive "Theory of Mind" model:
- Inverse Planning (The Audience): Borrowing from cognitive science, the authors model the audience as a Bayesian observer. When the audience sees an action, they infer the character's hidden goals by calculating $P(goal | action)$.
- Inverse Inverse Planning (The System): The animator (or the system) then treats the audience's inference as the objective function. Instead of maximizing the character's reward, the system chooses actions that maximize the audience's belief in a specific goal: $\text{argmax}_{action} P(desired\_inference | action)$.
Implementation Details: * Grid-World (The Kitchen): For discrete scenarios, the system uses beam search to find sequences of moves that maximize the audience's clarity regarding a robot's intent. * Physics-Based (The Hill): For continuous motion (e.g., a "Luxo-lamp" style hopper pulling a box), the system uses a differentiable physics simulator. It optimizes small deviations from an optimal policy to make a light box appear heavy—essentially teaching the AI to "mime."
Key Insights: A Unified Narrative Language
The authors show that a wide range of literary devices can be expressed as variations of the same Bayesian objective:
- Sub-optimality is Communicative: The most "expressive" action is often sub-optimal for the character's actual goal. A robot might move out of the way early to signal helpfulness, even if it delays its own progress.
- Plot Twists: Optimize for an initial belief in Goal A, followed by a sudden shift to Goal B.
- Irony: Optimize for a scenario where the character acts on a false belief that the audience knows is wrong.
- Flashbacks: Show a past action that only makes sense once a new context is revealed.
- Deus Ex Machina: The system can account for "outside" events (like a table falling) and optimize how characters react to maintain a specific narrative arc.
Key Results
Human subject studies provided strong quantitative evidence that I2P works:
- Clarity of Intent: In the "helping" task, viewers correctly identified intent 73% of the time with I2P, compared to only 6% with a "naïve" rational planner.
- The Mime Effect: In the physics world, 95.7% of viewers were convinced a light box was heavy because of the hopper's optimized "struggling" movements. Remarkably, 68.6% of viewers thought the "mimed" light box was heavier than an actual heavy box pulled by a standard rational agent.
Limitations and Future Work
The current model uses exact Bayesian inference, which is computationally expensive. Future work may move toward Sequential Monte Carlo (SMC) or neural approximations for real-time use. Additionally, the authors envision extending this to "emotions" and creating artist interfaces where animators can "sketch" narrative arcs rather than writing objective functions.
Summary updated