AlphaContext: An Evolutionary Tree-based Psychometric Context Generator for Creativity Assessment

Yixuan Wang, Yue Huang, Hong Qian, Yunzhao Wei, Yifei Ding, Wenkai Wang, Zhi Liu, Zhongjing Huang, Aimin Zhou, Jiajun GuoarXiv 2026 · arXiv

TL;DR

AlphaContext is a novel framework that automates the creation of high-quality, long-form psychometric scenarios used to assess human and AI creativity. By combining rule-guided tree planning, Monte Carlo Tree Search (MCTS), and evolutionary optimization (MAP-Elites), it generates diverse and coherent narratives embedded with subtle assessment cues, outperforming state-of-the-art LLMs by an average of 8% across key quality metrics.

What Problem Does This Paper Address?

Evaluating creativity scientifically is notoriously difficult. One of the most effective methods in psychometrics is context-based assessment, where participants are immersed in a complex, future-oriented scenario (e.g., navigating the ethical dilemmas of a ubiquitous "AI Partner") and asked to identify potential challenges or solutions.

Historically, crafting these scenarios has required scarce and expensive expert human craftsmanship. Standard Large Language Models (LLMs) fail at this task out-of-the-box; they struggle to maintain long-form narrative coherence, fail to subtly embed the necessary psychological "cues" that prompt creative thinking, and lack stylistic diversity. This paper introduces AlphaContext to bridge the gap between automated text generation and rigorous psychometric validity.

Figure 1: Workflow of creativity context assessment. Experts first design an “AI Partner” psychometric con- text containing implicit challenge cues. Participants then complete an open-ended task to identify potential challenges based on the context in their responses.

Methods

AlphaContext uses a four-stage "plan-generate-evolve-refine" pipeline:

  1. HyperTree Outline Planner: Instead of standard linear planning, expert reasoning is mapped into a rule-guided "hypertree." This structure breaks down the narrative top-down, jointly organizing the discourse structure and strategically placing implicit assessment cues.
  2. MCTS-based Context Generator: To fill out the outline, the system treats sentence generation as a tree search. It uses Monte Carlo Tree Search (MCTS) paired with a dual-horizon valuation mechanism to ensure that each sentence contributes to both local vividness and global narrative structure.
  3. Evolutionary Context Optimizer: To prevent the model from generating repetitive styles, the authors employ MAP-Elites, a quality-diversity evolutionary algorithm. It maps generated contexts into a 3D behavioral space defined by proximity scope (personal vs. public framing), knowledge density (objective evidence/data), and viewpoint diversity (number of stakeholders). It then iteratively mutates and updates the best text ("elites") within each niche.
  4. Assessment-Guided Evolution Refiner: To ensure the generated scenarios actually elicit creative thinking, AlphaContext simulates virtual participants with distinct response profiles (talkative, normal, quiet). An LLM scorer evaluates these simulated answers; if a context fails to provoke diverse, creative responses, it is recycled back into the evolutionary loop for further optimization.
Figure 2: The procedure of the proposed AlphaContext. (a) Given a context query Q, (b) the HyperTree Outline Planner and MCTS-based Context Generator generate seed contexts, and (c) the Evolutionary Context Optimizer and Assessment-guided Evolution Refiner improve diversity and quality, yielding assessment-ready contexts.

Datasets

The authors introduced CreaTE, a newly curated dataset consisting of 203 expert-curated title–theme pairs spanning diverse domains. Each entry was authored and iteratively cross-checked by three creativity-psychology experts to ensure the themes delineate clear problem spaces and creative tensions suitable for psychometric testing.

Key Insights and Ideas

  • Structure Meets Evolution: Pure prompting often traps LLMs in local semantic minima. Combining a rigid, rule-based tree structure (HyperTree/MCTS) with fluid evolutionary exploration (MAP-Elites) allows the system to maximize both structural compliance and stylistic creativity.
  • Simulation as a Validation Proxy: Testing psychometric tools usually requires expensive pilot studies with human subjects. By simulating participant personas (e.g., quiet vs. talkative), the framework creates a closed-loop validation mechanism that optimizes the text before any human lays eyes on it.
  • Implicit Cueing: Unlike standard storytelling, psychometric contexts must hide the answers within the text. AlphaContext successfully balances narrative engagement with the subtle planting of multi-dimensional challenges.

Key Results

AlphaContext was evaluated against 10 competitive baselines, including frontier models like GPT-5.1, Gemini-3.0-Pro-Preview, and DeepSeek-V3.1, as well as specialized long-form generation frameworks like DOC-v2 and SS-GEN.

  • Multi-Dimensional Superiority: AlphaContext ranked first across all 7 evaluation metrics, showing the largest gains in Coherence, Engagement, Significance, and Uncertainty (open-endedness).
  • Expert Alignment: In text-similarity matches against real, expert-deployed psychometric tests, AlphaContext achieved the highest scores in ROUGE-1 (30.41%), ROUGE-L (25.48%), and BERTScore (81.88%).
  • Real-World Human Validation: The authors conducted a live study with 36 secondary-school students. The creativity scores elicited by AlphaContext-generated scenarios demonstrated a statistically significant positive correlation ($r = 0.3770$) with the Alternative Uses Task (AUT), a gold-standard standardized creativity test, confirming its real-world psychometric validity.
Figure 3: Preference evaluation of AlphaContext vs. GPT-5.1 under human and DeepSeek-V3.1 judgments.

Limitations and Future Work

The primary limitation of AlphaContext is its computational overhead. Because it relies on sentence-level MCTS and iterative MAP-Elites mutations, it takes roughly 6 minutes and consumes significantly more tokens (~12.8k tokens) to generate a single context compared to standard zero-shot prompting (which takes seconds). However, the authors note that context generation is an offline task, and 6 minutes is a massive reduction compared to the week or more required for manual human expert design.

Future work will focus on expanding the CreaTE dataset to broader domains, adapting scenarios for different age groups and languages, and using AlphaContext outputs to fine-tune lighter, more efficient generator models.

Summary updated

Abstract

Creativity has become a core competence in the era of LLMs and human-AI collaboration, underpinning innovation in real-world problem solving. Crucially, the systematic improvement of creativity necessitates scientifically valid assessment instruments. Psychometric research recognizes context-based assessment as an effective way to measure creative thinking. However, high-quality expert-designed contexts remain scarce. Existing LLM-based generators often struggle with insufficient assessment cues, weak narrative coherence, limited stylistic diversity, and poor support for creative thinking. To address these challenges, we propose AlphaContext, an evolutionary tree-based psychometric context generator for creativity assessment. First, the HyperTree Outline Planner formalizes expert-designed outlining as a rule-guided hypertree and performs top-down hierarchical planning. The MCTS-based Context Generator fills the outline via MCTS to balance global structure and local quality. Then, the Evolutionary Context Optimizer evolves contexts with MAP-Elites by repeatedly updating niche elites to jointly improve diversity and quality. Finally, the Assessment-Guided Evolution Refiner simulates virtual participants with diverse styles and recycles weak contexts for further evolution. Experiments show that AlphaContext yields an average improvement of 8% over competitive methods across 6 quality metrics.
BibTeX
@misc{wang2026alphacontext,
  author = {Yixuan Wang and Yue Huang and Hong Qian and Yunzhao Wei and Yifei Ding and Wenkai Wang and Zhi Liu and Zhongjing Huang and Aimin Zhou and Jiajun Guo},
  title = {{AlphaContext: An Evolutionary Tree-based Psychometric Context Generator for Creativity Assessment}},
  year = {2026},
  eprint = {2604.18398},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2604.18398},
}
Figure