Shaping AI's Beliefs Before It Generates: Human Sense of Agency in Creative AI

Zi Wang, Mahima PushkarnaarXiv 2026 · arXiv

This paper from researchers at Google DeepMind introduces the Proactive Co-Creator (PaCC), a system designed to fix a growing problem in generative AI: the "anchoring effect." By allowing users to inspect and edit an AI’s internal assumptions before it generates a single pixel or sentence, the authors aim to restore a human sense of agency that is often lost in "prompt-and-pray" workflows.

TL;DR

The researchers propose "pre-generation belief control" to prevent users from becoming reactive editors of polished AI artifacts. Their system, PaCC, surfaces the AI's underlying conceptual interpretations through editable graphs and clarification questions, significantly boosting users' sense of process control and agency.

What Problem Does This Paper Address?

Current generative AI interactions typically follow a "post-generation critique" loop: you provide a prompt, the AI generates a high-fidelity result, and you then iterate by editing that result.

The authors argue this creates an anchoring effect. Because the initial output is so polished, the user’s creative search space contracts around the AI’s choices. Instead of being an active author, the user becomes a reactive curator. This leads to "intent debt"—a disconnect where the final output reflects the machine's defaults more than the human's original vision—and "cognitive surrender," where users uncritically accept whatever the model spits out.

Methods

The authors formalized four design principles for "pre-generation belief control":

  1. Intervene before generation (DP1): Shift iteration from post-hoc editing to upstream conceptual guidance.
  2. Make assumptions visible (DP2): Surface the AI's internal "beliefs" about a prompt.
  3. Preserve the possibility space (DP3): Offer diverse options rather than singular defaults.
  4. Calibrate AI autonomy (DP4): Let users decide how much "creative liberty" the AI should take.

They instantiated these in PaCC, a multi-modal system built using the Gemini API. As shown in the interface below, PaCC breaks down a prompt into three steerable components:

Figure 1: The Proactive Co-Creator (PaCC) interface for shaping AI’s beliefs before generation (§2). (A) Prompt entry with optional reference images. Creators control AI proactiveness (A1) across three tiers: literal extraction (low), reasonable scene inference (medium; e.g., adding a “party table” to the cat party), or creative liberty (high; e.g., a “disco ball”). Modality tabs switch between Image, Story, and Video (A2). (B) Belief Graph: An editable visualization of explicit entities (solid borders), AI-inferred entities (dashed borders), and relationship edges (e.g., “hosting”). (C1–C2) Clarification Questions: Offers 3–5 selectable options per question so creators can resolve ambiguity and guide the AI’s interpretation; clicking “You Decide” delegates a specific choice to the AI. (D1–D2) Entity Attribute Editing: Adjusts properties of individual nodes. (E1–E2) Media Attributes: Controls global scene aesthetics (e.g., lighting, style, pacing). By surfacing and aligning assumptions and possibilities prior to media generation, PaCC establishes the human intent as the anchor.
  • Clarification Dialogues: The AI proactively asks questions (e.g., "What is the specific theme for the cat's party?") to resolve ambiguity.
  • Belief Graphs: An editable visualization of entities and their relationships. It distinguishes between "explicit" entities (from the prompt) and "implicit" ones (inferred by the AI).
  • Media Attributes: Global controls for style, lighting, and pacing that expose the AI's stylistic defaults.

Datasets

The paper does not introduce a new training dataset. Instead, it utilizes the Gemini family of models (3.1 Pro, Flash, etc.) via API calls. The evaluation is based on an exploratory user study with N=8 participants (AI researchers, designers, and UX professionals) who used the tool for self-directed creative tasks.

Key Insights and Ideas

  • Surfacing the "Unspoken": Participants noted that seeing the AI's assumptions helped them think through details they had overlooked. The AI’s "beliefs" acted as a mirror for their own incomplete intent.
  • The Ownership Paradox: Interestingly, while "Sense of Agency" and "Control" scores were high, "Ownership" and "Authorship" were more polarized. Some users felt that choosing from a menu of AI-generated options made them feel less like the author, even if they had more control over the process.
  • Literal vs. Creative: The "Proactiveness Tiers" allowed users to toggle between literal extraction (low) and creative liberty (high). This calibration is crucial for different stages of the creative process—from strict prototyping to wild brainstorming.
  • Relational over Stylistic: Users found it easier to map relational concepts (e.g., "the cat is hosting the bunny") in the belief graph than to specify visual styles through text alone.

Key Results

The study used a 5-point Likert scale (1=diminished, 3=preserved, 5=augmented) to measure the impact of PaCC compared to standard prompt-based tools.

Table 1: Mean Likert ratings (M ± SD on a 1–5 scale; 3=preserved) and the number of participants rating each PaCC feature as providing Greater/Equal/Lesser (G/E/L) agency, ownership, control, or authorship than their current AI tools. N for G/E/L varies because some participants selected “N/A.”
  • Overall Sense of Agency (SoA): Rated at 4.09 ± 0.84, indicating a strong positive shift.
  • Process Control: Rated at 3.89 ± 0.69.
  • Clarification Questions: This was the most successful feature; all 8 participants rated it as providing greater agency than their current tools.
  • Authorship: This was the lowest-rated dimension (2.75 ± 1.13), suggesting that simply providing more "knobs" to turn doesn't automatically make a user feel like the "artist."

Limitations and Future Work

The authors acknowledge several limitations:

  • Small, Technical Sample: The study only involved 8 people, mostly from technical backgrounds. Professional artists might have very different "agency thresholds."
  • Text-Only Style Control: Users struggled to specify complex visual styles using only text-based attributes and requested multimodal inputs like sketches or reference images.
  • Short Duration: The 30-minute sessions captured "honeymoon" reactions; longitudinal studies are needed to see if these controls become tedious over time.

Future work aims to integrate multimodal style steering (e.g., "make it look like this sketch") and explore how AI reward models can be trained to incentivize "collaborative negotiation" rather than just high-quality one-shot outputs.

Summary updated

Abstract

AI systems for creative media generation expand human capabilities yet risk diminishing human agency. A core problem is anchoring, where the user's thinking becomes anchored to a polished, generated artifact, shifting the user's role from active authoring to reactive editing. We propose pre-generation belief controls that can empower creators to inspect, revise, and steer the AI's interpretation before any media is generated, so that the initial anchor is set by the human. We formalize four design principles and instantiate them in the Proactive Co-Creator (PaCC), a system for early-stage conceptual ideation and prototyping that proactively surfaces its assumptions and uncertainty through clarifications, belief graphs, and media attributes, inviting users to shape the AI's beliefs before generating the final outcome. Using PaCC as a design probe in a small-scale, single-arm exploratory user study, we observed that participants reported directionally positive sense of agency and process control relative to their familiar prompt-based AI tools, noting that surfaced assumptions helped them think through details they had overlooked, whereas their feelings about ownership and authorship diverged
BibTeX
@misc{wang2026shaping,
  author = {Zi Wang and Mahima Pushkarna},
  title = {{Shaping AI's Beliefs Before It Generates: Human Sense of Agency in Creative AI}},
  year = {2026},
  eprint = {2611.21405},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.21405},
}
Figure