Human-AI Interaction Generation: A Connective Lens for Generative AI and Procedural Content Generation

Matthew GuzdialIJCAI 2024

Abstract This paper, presented at IJCAI-24, proposes a unified "Human-AI Interactive Generation" (HAIG) framework to bridge the gap between modern Generative AI (GenAI) and traditional Procedural Content Generation (PCG). By formalizing the relationship between these fields, the research identifies three critical attributes—human-centered input alignment, adaptability, and novelty—that are essential for evolving AI from "black-box" generators into collaborative creative partners.

Introduction: The Convergence of GenAI and PCG

The current landscape of Generative AI is dominated by "single-shot" interactions, where users provide a prompt and receive a final output. While powerful, this paradigm often fails to support complex, iterative creative processes. Conversely, the field of Procedural Content Generation (PCG) has decades of experience in rule-based and search-based generation for games but is often siloed from modern deep learning advances. This paper argues that this divide is artificial and proposes a connective lens to synthesize these disciplines into a single field of Human-AI Interactive Generation.

A Unified Functional Framework

The relationship between these fields can be formalized using a simple functional lens:

$$Y = G(X)$$

  • $Y$ (Possibility Space): The set of all possible outputs a generator can produce.
  • $G$ (Generator Space): The space of all possible generators (e.g., different versions of an LLM or a rule-based script).
  • $X$ (Input Space): The set of all possible inputs (prompts, noise, or context).

This framework allows researchers to pinpoint where current tools fall short: the "alignment gap," where inputs do not match human workflows, and the "static gap," where models fail to learn from specific users during a session.

Three Pillars of Human-AI Interactive Generation

To bridge these gaps, the paper identifies three novel approaches derived from the author's prior research:

  1. Human-centered Input Alignment: Designing the input set ($X_g$) to match human expectations. For example, the Sketchbetween tool uses artist-drawn sketches rather than text prompts to better align with how animators actually work.
  2. Adaptability: Creating models where the generator $g$ evolves in real-time based on a sequence of user interactions $(x_0, x_1, ... x_n)$. In studies with the Morai Maker level editor, users consistently preferred agents that adapted to their specific style over technically "accurate" static models.
  3. Novelty via Conceptual Expansion: A method to generate "out-of-distribution" content by treating generators as a linear combination of features and filters: $CE(F, A) = \alpha_1 * f_1 + \alpha_2 * f_2...$. This allows the system to reach parts of the possibility space that individual models cannot access alone.

Case Studies and Empirical Results

The effectiveness of this framework is demonstrated across several specialized platforms:

  • Sketchbetween: This tool outperformed general image inpainting baselines because its input modality was specifically aligned with animation workflows.
  • Conceptual Expansion: In human subject studies, games generated via this method were ranked as more surprising than those authored by human experts, while maintaining high perceived creativity.
  • The Feedback Paradox: Research using the Anhinga puzzle tool revealed that an AI responding to every user change caused cognitive overload. Users actually preferred less reactive systems, suggesting that the frequency of interaction must be carefully balanced.
Figure 1: Visual results from the Sketchbetween tool and Conceptual Expansion experiments, showing the generation of novel character designs from human-aligned sketch inputs.

Discussion and Future Directions

While the research demonstrates the value of alignment, adaptability, and novelty, several challenges remain. The current research is somewhat fragmented, and there is no unified theory for choosing the optimal input modalities for specific tasks. Furthermore, the "when" of adaptation is critical; future work must determine the precise timing and degree to which an AI should change its behavior to avoid confusing the user.

Ultimately, the paper argues that no AI is truly autonomous, as every process requires a human catalyst. By focusing on these three pillars, the field can move toward creating AI partners that truly understand and augment human creativity rather than just replacing it.

Summary updated

BibTeX
@inproceedings{guzdial2024human,
  author = {Matthew Guzdial},
  title = {{Human-AI Interaction Generation: A Connective Lens for Generative AI and Procedural Content Generation}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2024},
}

Loading related content...

Figure