Abstract
SCoTER (Structured Chain-of-Thought Transfer) is a framework designed to bridge the gap between the sophisticated reasoning capabilities of Large Language Models (LLMs) and the strict latency requirements of production recommender systems. By introducing a Generate-Validate-Mine (GVM) pipeline, SCoTER automatically discovers dataset-specific reasoning patterns, moving beyond brittle handcrafted templates. These patterns are then distilled into a structured, order-aware architecture that preserves the sequential logic of a "Chain-of-Thought" (CoT) without the overhead of online LLM calls. Evaluations on multiple benchmarks and a large-scale advertising platform demonstrate that SCoTER significantly improves recommendation accuracy, particularly in cold-start scenarios, while maintaining millisecond-level inference speeds.
Introduction
Integrating Large Language Models (LLMs) into recommender systems offers the promise of "reasoning"—the ability to explain why a user might prefer an item based on their historical interactions. However, two primary hurdles remain: the "What to Transfer" challenge, where handcrafted reasoning templates often fail to capture domain-specific nuances, and the "How to Transfer" challenge, where direct LLM inference is too slow for real-time systems and standard distillation methods often "flatten" complex logic into single vectors. SCoTER addresses these issues by automating pattern discovery and employing a structured integration approach.
The SCoTER Framework
The SCoTER framework overview, illustrating the transition from CoT Pattern Discovery to Structured CoT Integration. Source, p. 2
The SCoTER architecture is divided into two main phases: automated pattern discovery and structured integration. This separation allows the system to leverage the reasoning power of a "teacher" LLM offline while deploying a highly efficient "student" model and recommendation backbone online.
Automated Pattern Discovery: The GVM Pipeline
The Generate-Validate-Mine (GVM) pipeline identifies high-quality reasoning patterns by validating LLM-generated paths against ground-truth data and clustering successful templates. Source, p. 4
To solve the "What to Transfer" problem, SCoTER utilizes the Generate-Validate-Mine (GVM) pipeline. Instead of relying on manual prompts, the system uses an LLM to generate diverse candidate reasoning paths. These paths are validated against actual ground-truth user data to identify which logical sequences lead to the most accurate recommendations. Finally, the most successful paths are mined using K-Means clustering to extract a "master" reasoning template optimized for the specific dataset. This process ensures that the reasoning logic is tailored to the domain, such as distinguishing between long-term preferences in "Beauty" versus identifying complementary gear in "Sports."
Structured CoT Integration
Once optimal patterns are discovered, SCoTER employs a structured distillation and fusion process to solve the "How to Transfer" problem:
- Distillation: A powerful teacher model (e.g., DeepSeek-R1) generates reasoning chains based on the discovered templates, which are then used to fine-tune a smaller, more efficient student model (e.g., Qwen3-8B).
- Order-Preserving Fusion: To prevent the "flattening" of logic, SCoTER represents reasoning chains as a matrix of step-embeddings. Using cross-attention and positional encodings, the recommendation backbone (such as TIGER or HSTU) is forced to respect the sequential flow of the logic, ensuring that each step of the reasoning (e.g., "Analyze history") actually informs the subsequent steps and the final recommendation.
Experimental Evaluation
SCoTER was evaluated across four Amazon product review subsets, Yelp, and a massive real-world production environment at Tencent.
- Offline Performance: SCoTER consistently outperformed state-of-the-art baselines. Notably, on the Amazon Beauty dataset, it achieved a 27.63% relative improvement in Recall@10 over the strong HSTU baseline.
- Online Impact: In a live A/B test on the Tencent Advertising Platform, the framework delivered a 2.14% lift in Gross Merchandise Value (GMV).
- Efficiency: By moving LLM reasoning to an offline distillation and pre-computation phase, SCoTER maintains the millisecond-level latency required for production environments, eliminating online LLM inference costs entirely.
Insights and Future Directions
A key finding of this work is that explicit logical reasoning acts as a "sparse signal booster," providing the most significant gains for "cold-start" users with few interactions. Furthermore, the authors demonstrated that the sequential structure of reasoning is non-negotiable; methods that collapse the chain (e.g., mean pooling) significantly degrade performance.
Future work will explore making reasoning patterns dynamic and user-specific in real-time, as well as expanding the framework to include multi-modal "thoughts," such as reasoning about visual styles or auditory features.
Conclusion
SCoTER provides a robust pathway for bringing the reasoning depth of LLMs to high-throughput recommender systems. By automating the discovery of logic and preserving its structure during transfer, SCoTER achieves a balance of high accuracy and production-ready efficiency.
Summary updated