Abstract
Large Language Models (LLMs) offer immense potential for content creation, yet non-expert users often struggle with "prompt engineering" and the cognitive burden of identifying necessary details for specific tasks. This paper presents HELP ME THINK, a user-centric prompting strategy that flips the traditional interaction model. Instead of requiring complex instructions, the model adopts an expert persona to ask the user clarifying questions, then synthesizes the answers into high-quality customized content. Evaluation across diverse creative tasks demonstrates high relevance, coherence, and effective knowledge absorption.
Introduction: Overcoming the Thinking Burden
While LLMs are capable of generating professional bios or travel itineraries, the average person often faces "writer's block" when trying to provide the exact instructions and constraints required for high-quality output. This "thinking burden" stems from the user needing to know what information is necessary before they can even begin prompting.
Existing techniques like few-shot prompting or negative constraints are often too complex for casual users. HELP ME THINK addresses this by automating the information-gathering phase, making AI more accessible by guiding the user through the creative process.
Methodology
The researchers propose a model-agnostic, three-stage framework:
- Generate Questions: The model adopts an expert persona (e.g., "expert travel planner") and is instructed to ask the user clarifying questions to gather necessary data.
- Collect Answers: The user provides conversational responses. In experimental settings, crowdworkers simulated real-world usage to provide these answers.
- Generate Task-Specific Output: The model is prompted with the original goal and the collected question-answer pairs to synthesize the final product.
Data and Evaluation
To evaluate the strategy, a new dataset was constructed covering six creative domains: bio generation, travel planning, dialogue, poetry, event summaries, and story generation. The dataset comprises approximately 2,000 question-answer pairs and 180 task-specific outputs. The method's versatility was further tested on 57 additional tasks, ranging from bridge construction planning to sports team formation.
Key Results
Human evaluation revealed that the HELP ME THINK strategy significantly improves the quality and relevance of AI-generated content.
Key performance metrics include:
- 100% Relevance: All model-generated questions were found to be valid and relevant to the task.
- 99% Coherence: The model successfully generated coherent sentences, adding appropriate context beyond the raw user input.
- 70% Knowledge Absorption: On average, the model successfully transferred the majority of user-provided facts into the final output.
- 94% Fact Accuracy: Most outputs contained no hallucinations or irrelevant information beyond the user's provided data.
Insights and Discussion
The core insight is that for personalized content, the AI's ability to ask the right questions is as valuable as its ability to generate text. By breaking complex tasks into manageable pieces, the model reduces the user's cognitive load.
Interestingly, the study found that LLMs (specifically GPT-3) act as natural polishers, correcting typos and grammar in user answers in roughly 83% of cases. However, there is a trade-off: providing too many details at once can decrease "knowledge absorption," suggesting that decomposing tasks into smaller chunks or fewer questions can improve final accuracy.
Limitations and Future Work
While promising, the 70% retention rate indicates room for improvement in handling large volumes of facts. Future research should explore multi-turn dialogues for deeper refinement, expand the strategy to non-English languages, and develop automated frameworks for large-scale stress-testing across niche domains.
Summary updated