TL;DR
This paper introduces DASH, a decision-aware user simulator for online advertising that goes beyond simply mimicking user actions (like clicks) to jointly generating the underlying "thinking traces" behind those actions. By leveraging heterogeneous cross-domain interaction histories and a progressive three-stage framework (Context Engineering, Supervised Fine-Tuning, and Reinforcement Learning), DASH achieves state-of-the-art simulation fidelity and provides valuable diagnostic insights for offline recommendation evaluation.
What Problem Does This Paper Address?
Evaluating new algorithms in online advertising via live A/B testing is expensive, risky, and can degrade user experience. While Large Language Model (LLM)-based user simulators offer a promising offline alternative, existing approaches suffer from two major flaws:
- Single-Domain Preference Modeling: They only look at interaction histories within a single domain, missing the broader context of a user's multi-domain digital footprint.
- Action-Only Response Modeling: They focus solely on predicting the final action (e.g., click vs. skip). This allows models to take "shortcuts" (overfitting to specific surface features) and fails to explain why a user made a choice, stripping the simulator of diagnostic value.
Tackling these flaws introduces two core challenges: compressing massive, noisy cross-domain histories into an LLM's context window without losing critical signals (Challenge 1), and supervising "thinking traces" when no ground-truth user thoughts exist in real-world logs (Challenge 2).
Methods
The authors propose DASH (Decision-Aware Simulator with Heterogeneous context), which uses a progressive three-stage pipeline:
- Stage 1: Context Engineering (CE): To handle millions of historical tokens, DASH uses a Hierarchical Context Compression strategy. At the item level, it keeps full details for high-information actions (clicks, conversions) but retains only category metadata for low-information actions (skips). At the stream level, it prioritizes the ad stream and truncates the general content stream. It also employs a closed-loop prompt optimization process to refine a structured "Focus $\rightarrow$ Draft $\rightarrow$ Verify $\rightarrow$ Finalize" reasoning chain.
- Stage 2: Supervised Fine-Tuning (SFT): To give a smaller, deployable student model (Qwen3.5-35B) basic reasoning capabilities, DASH distills thinking trajectories from a powerful teacher model (Qwen3.5-397B). It curates both "easy" samples (where the teacher naturally predicts the correct action) and "hard" samples (where the teacher is forced to regenerate a reasoning trace conditioned on the true action).
- Stage 3: Reinforcement Learning (RL): Using Group Relative Policy Optimization (GRPO), DASH aligns the student model using a hybrid reward function. This combines a sparse action fidelity reward with a dense, rubric-based thinking quality reward that scores the generated text across three dimensions: Form, Content, and Logic.
Datasets
The model was trained and evaluated on a large-scale, real-world cross-domain advertising dataset collected from the Tencent platform.
- The dataset includes user profiles, ad creative profiles, ad request contexts, and a 30-day interaction history.
- The interaction history spans five heterogeneous content domains: Apps, Games, Videos, Articles, and Search.
- On average, each ad request is accompanied by a massive raw context of roughly 3.73 million tokens before compression.
Key Insights and Ideas
- Logic Over Content: An intriguing finding was that the logical coherence of a user's simulated thinking trace correlates much more strongly with accurate action prediction than factual content density alone. Models that generated highly plausible content but lacked causal consistency performed poorly at predicting final actions.
- The "Lost-in-the-Middle" Remedy: Standard LLMs often drop critical historical signals when buried deep inside a long prompt. DASH's explicit "Focus" stage acts as an information retriever, pulling out and timestamping strong negative signals (like explicit negative feedback) before the reasoning process begins.
- RL as a Shortcut Preventer: Relying purely on SFT data limits the model to the teacher's coverage. Introducing trace-level RL rewards prevents the model from taking superficial shortcuts, ensuring it doesn't fabricate unobserved user needs to justify a predicted click.
Key Results
DASH significantly outperformed both its base model and massive general-purpose LLMs (including 1T parameter models like Kimi-K2.5) across action prediction and thinking quality metrics.
- Action Fidelity: DASH (Small-scale SFT + RL) achieved a Weighted F1 score of 62.15%, outperforming the teacher model (Qwen3.5-397B at 60.23%) and the baseline student model (55.18%).
- Thinking Quality: DASH achieved the highest overall logic score (92.62) and overall thinking quality score (92.01).
- Distribution Alignment: As shown below, the optimized prompt structure drastically reduced the vanilla model's tendency to over-predict clicks and under-predict conversions, aligning closely with real-world action frequencies.
Limitations and Future Work
While highly effective, the authors note that deploying LLM-based simulators at scale introduces dual-use risks, such as the potential for ad networks to over-optimize aggressive ranking or targeting strategies against simulated users. To mitigate this, they restrict DASH to evaluative and diagnostic use cases.
Future work focuses on integrating DASH directly into production infrastructure for three key applications: offline A/B test pre-screening, pre-testing ad creatives before they are served to real users, and optimizing the balance between platform monetization and user experience.
Summary updated