Exploring the Potential of Large Language Models in Artistic Creation: Collaboration and Reflection on Creative Programming

Anqi Wang, Zhizhuo Yin, Yulu Hu, Yuanyuan Mao, Pan Hui

Abstract

Large language models (LLMs) are increasingly used to assist in programming, yet their role in the nuanced domain of creative coding remains under-explored. This study investigates how different human-AI collaboration methods—invoking an entire program versus decomposing it into subtasks—affect the reflective processes of artists. By analyzing the creative journeys of 22 practitioners, we find that task decomposition significantly enhances the frequency and quality of reflection, shifting the focus from mere debugging to strategic creative transformation.

Introduction: Beyond Productivity in Human-AI Co-Creation

Current research on AI-assisted programming often prioritizes speed and technical accuracy. However, in creative coding, the journey is as important as the destination. Creative work is inherently iterative, requiring artists to constantly evaluate their goals and reframe their strategies. If an AI tool provides a "perfect" solution too quickly, it may inadvertently bypass the critical "reflective" moments necessary for artistic growth.

This paper addresses this gap by examining how two prompting strategies—Condition C1 (Entire Task) and Condition C2 (Decomposed Subtasks)—influence user reflection.

Methodology: Categorizing the Creative Mind

The researchers conducted a mixed-method study with 22 art and design practitioners using Code-Llama-Instruct-34B to create interactive visuals in P5.js. To quantify the "thinking" process, the team employed Baumer’s reflection framework, categorizing behaviors into three distinct types: Breakdown, Inquiry, and Transformation.

Table 1: Definitions and examples of the three reflection categories used to analyze participant behavior.

As defined in the table above, Breakdown is a reactive response to errors, while Inquiry and Transformation represent more proactive and generative cognitive states.

Results: Comparing Prompting Strategies

The study revealed a significant difference in how participants engaged with the AI depending on the prompting method. Participants in the decomposed subtask condition (C2) reflected significantly more often ($N=222$) than those in the whole-task condition (C1, $N=147$).

Figure 1: Mean frequency of reflection types across the two experimental conditions. Condition C2 (subtasks) consistently stimulated more reflection than C1 (entire task).

Statistical analysis (ART ANOVA) confirmed that C2 led to significantly more Breakdown ($p < .01$) and Inquiry ($p < .05$) reflections. While Breakdown reflections were the most common across both groups, the C2 group was better equipped to move beyond simple fixes toward deeper Inquiry.

Discussion: Decomposition as Cognitive Scaffolding

The findings suggest that breaking a complex creative goal into smaller, manageable subtasks acts as a form of "cognitive scaffolding."

In the Entire Task (C1) condition, users often fell into a "reactive repair" trap. When the LLM failed to grasp a complex, multi-layered prompt, users entered frustrating loops of rephrasing without rethinking their logic. This often led to "affective struggle," where artists simplified or abandoned their original creative visions.

Conversely, the Decomposed Subtasks (C2) condition forced users to engage in "Inquiry." By managing smaller logic units, users could test hypotheses and adjust parameters more effectively. This "productive struggle" allowed for "compressed metacognition," where artists rapidly synthesized AI feedback with their own prior knowledge to make agile creative decisions.

Conclusion and Future Work

This research highlights that for AI to be a true creative partner, it must support—not just replace—human reflection. Task decomposition is a powerful strategy for fostering this engagement. Future AI tools might benefit from "reflection-supportive modes" that detect user cognitive states and provide prompts that encourage Inquiry and Transformation rather than just providing immediate code solutions.

While limited to the domain of P5.js and short-term interactions, this study provides a critical lens for the design of future human-AI collaborative systems in the arts.

Summary updated

Abstract

Recently, the potential of large language models (LLMs) has been widely used in assisting programming. However, current research does not explore the artist potential of LLMs in creative coding within artist and AI collaboration. Our work probes the reflection type of artists in the creation process with such collaboration. We compare two common collaboration approaches: invoking the entire program and multiple subtasks. Our findings exhibit artists' different stimulated reflections in two different methods. Our finding also shows the correlation of reflection type with user performance, user satisfaction, and subjective experience in two collaborations through conducting two methods, including experimental data and qualitative interviews. In this sense, our work reveals the artistic potential of LLM in creative coding. Meanwhile, we provide a critical lens of human-AI collaboration from the artists' perspective and expound design suggestions for future work of AI-assisted creative tasks.
BibTeX
@misc{wang2024exploring,
  author = {Anqi Wang and Zhizhuo Yin and Yulu Hu and Yuanyuan Mao and Pan Hui},
  title = {{Exploring the Potential of Large Language Models in Artistic Creation: Collaboration and Reflection on Creative Programming}},
  year = {2024},
  eprint = {2402.09750},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2402.09750},
}
Figure