Supporting Effective Goal Setting with LLM-Based Chatbots

Michel Schimpf, Sebastian Maier, Anton Wyrowski, Lara Christoforakos, Stefan Feuerriegel, Thomas Bohné

Abstract

Each day, individuals set behavioral goals such as eating healthier, exercising regularly, or increasing productivity. While psychological frameworks (i.e., goal setting and implementation intentions) can be helpful, they often need structured external support, which interactive technologies can provide. We thus explored how large language model (LLM)-based chatbots can apply these frameworks to guide users in setting more effective goals. We conducted a preregistered randomized controlled experiment ($N = 543$) comparing chatbots with different combinations of three design features: guidance, suggestions, and feedback. We evaluated goal quality using subjective and objective measures. We found that, while guidance is already helpful, it is the addition of feedback that makes LLM-based chatbots effective in supporting participants' goal setting. In contrast, adaptive suggestions were less effective. Altogether, our study shows how to design chatbots by operationalizing psychological frameworks to provide effective support for reaching behavioral goals.

Introduction

A significant challenge in personal development is the difficulty individuals face when attempting to set effective, actionable goals. Most people tend toward vagueness (e.g., "I want to get fit") rather than specificity. While psychological frameworks like Goal Setting Theory—which emphasizes specificity and challenge—and Implementation Intentions (the "if-then" planning for action) are empirically proven, they are difficult for individuals to apply without professional coaching.

While human coaching is effective, it lacks the scalability required for broad public health or productivity interventions. Previous generations of chatbots were often too rigid to provide the nuanced, context-aware support necessary for behavior change. This study investigates whether Large Language Models (LLMs) can bridge this gap by acting as scalable, interactive goal-setting assistants.

Methods

We conducted a preregistered randomized controlled experiment with 543 participants recruited via Prolific. The study utilized a custom iOS-based chatbot platform powered by gpt-4o. Participants were randomly assigned to one of five conditions designed to isolate the impact of specific design features:

  1. ControlBot: A static interface mimicking a digital form, presenting theory and asking for a goal.
  2. GuidanceBot: A rule-based system leading users through a step-by-step process (e.g., defining goal type and difficulty).
  3. SuggestionBot: Added LLM-generated, personalized examples to the guidance (e.g., "If you want to run a marathon, you could try...").
  4. FeedbackBot: Added an LLM-driven feedback loop that critiqued user input and requested missing details.
  5. GenBot: A comprehensive version combining guidance, suggestions, and feedback.
Overview of the study design, including chatbot features (A), experimental conditions (B), interaction flow (C), and evaluation metrics (D).

The resulting goals and action plans were objectively rated by expert coders, while participants provided self-reports on their commitment and the perceived "social presence" of the chatbot.

Results

Feedback as the "Killer Feature"

The study identified interactive feedback as the primary driver of goal quality. Simply providing guidance or suggestions was insufficient; the chatbot needed to actively "push back" and request more detail to significantly improve specificity. Participants using the GenBot and FeedbackBot set significantly more specific goals than those in the control or guidance-only groups ($p < .001$). The quality of "if-then" action plans was also vastly higher in feedback conditions, with the FeedbackBot achieving a substantial effect size ($r = 0.50$) over the GuidanceBot.

Example of a chatbot interaction where the system provides feedback to refine a vague goal into a specific plan.

The Quality-Motivation Trade-off

A surprising finding was the inverse relationship between goal quality and immediate user commitment. As goals became more specific and scientifically rigorous, users' self-reported commitment actually decreased. This suggests that making a goal concrete makes the associated effort and "cost" more salient, which can be demotivating in the short term.

Social Presence and Suggestions

LLMs elicited a higher sense of "social presence" compared to rule-based bots. While this social presence increased commitment, it was ultimately outweighed by the demotivating effect of the rigorous goal-setting process. Furthermore, adaptive suggestions (personalized examples) did not significantly improve goal quality compared to simple step-by-step guidance.

Discussion and Limitations

This study demonstrates that LLM-based feedback is highly effective at improving the formulation of goals. However, the observed "commitment paradox" highlights a critical design challenge: how to provide rigorous structure without dampening user motivation.

A primary limitation is the focus on goal formulation rather than long-term attainment. It remains to be seen if the higher-quality goals produced by LLMs lead to better behavioral outcomes despite lower initial commitment. Future research should explore designs that maintain user autonomy and use behavioral markers—such as actual task completion—to measure success.

Summary updated

Abstract

Each day, individuals set behavioral goals such as eating healthier, exercising regularly, or increasing productivity. While psychological frameworks (i.e., goal setting and implementation intentions) can be helpful, they often need structured external support, which interactive technologies can provide. We thus explored how large language model (LLM)-based chatbots can apply these frameworks to guide users in setting more effective goals. We conducted a preregistered randomized controlled experiment ($N = 543$) comparing chatbots with different combinations of three design features: guidance, suggestions, and feedback. We evaluated goal quality using subjective and objective measures. We found that, while guidance is already helpful, it is the addition of feedback that makes LLM-based chatbots effective in supporting participants' goal setting. In contrast, adaptive suggestions were less effective. Altogether, our study shows how to design chatbots by operationalizing psychological frameworks to provide effective support for reaching behavioral goals.
BibTeX
@misc{schimpf2026supporting,
  author = {Michel Schimpf and Sebastian Maier and Anton Wyrowski and Lara Christoforakos and Stefan Feuerriegel and Thomas Bohné},
  title = {{Supporting Effective Goal Setting with LLM-Based Chatbots}},
  year = {2026},
  eprint = {2602.08636},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2602.08636},
}
Figure