TL;DR
CutAgent is a dual-agent framework that automates the creation of high-conversion e-commerce videos by decoupling creative strategy from technical execution. By introducing a structured "creative gene" and a self-refining reflection loop that learns from past production failures, it achieves a record-breaking online click-through rate (CTR) of 2.21%.
What Problem Does This Paper Address?
Creating effective e-commerce advertising videos is a balancing act between high-level creative planning (what's the hook? what are the selling points?) and low-level production (clip editing, TTS, music matching).
Current automated pipelines typically map product attributes directly to scripts or editing decisions. This "black box" approach has two major flaws:
- Lack of Strategy: It ignores the global advertising strategy, making it difficult to optimize the video based on commercial feedback like CTR.
- Fragility: These systems are error-prone—often missing key selling points or producing mismatched audio—and they lack a mechanism to "learn" from these failures to avoid repeating them in future videos.
Methods
The authors propose a Plan–Execute–Self-Evolve paradigm realized through two collaborative agents:
1. GeneAgent (The Planner)
GeneAgent acts as the creative director. It takes multi-modal product inputs (textual attributes and images) and maps them to a Creative Gene ($\widetilde{G}_i$). This "gene" is a structured representation containing:
- Narrative Units: An ordered sequence of goals (e.g., "Attention Hook" $\rightarrow$ "Feature Demo" $\rightarrow$ "Trust Building").
- Production Rules: Constraints for script style, voice-over tone, and music.
To optimize for conversion, GeneAgent is first fine-tuned on high-CTR historical data and then further refined using Group Relative Policy Optimization (GRPO), guided by a surrogate reward model that predicts CTR.
2. VideoAgent (The Executor & Evolver)
VideoAgent handles the heavy lifting of production. Guided by the creative gene, it:
- Retrieves Experiences: It queries a long-term memory for "failure-correction" pairs from similar product categories to avoid past mistakes.
- Coordinates Production: It manages shot-script planning, clip retrieval, TTS/subtitle generation, and BGM matching.
- Self-Refinement: After rendering, an independent reflection loop evaluates the video across six dimensions (e.g., narrative quality, selling-point coverage). If defects are found, it revises the decisions and writes the verified correction back to the long-term memory.
Datasets
The researchers introduced two significant resources:
- CutAgent-70K: A massive dataset of 70,000 samples. Each sample includes product attributes, images, a "reverse-extracted" creative gene from a real ad, a pool of cleaned video clips, and real-world CTR data.
- CutAgent-Eval: A benchmark of 5,000 products across 43 categories, used to compare CutAgent against state-of-the-art baselines.
Key Insights and Ideas
- The "Creative Gene" as an Interface: By creating a structured intermediate representation, the authors decoupled the intent of the ad from the rendering. This allows the system to optimize for high-level marketing strategies using reinforcement learning (GRPO) without getting bogged down in pixel-level details.
- Learning from Failure: Most agents are "one-and-done." CutAgent’s use of a long-term memory for failure-correction experiences allows it to perform non-parametric self-evolution. It effectively builds a "knowledge base" of what works and what doesn't for specific product categories.
- Surrogate CTR Rewards: Since real-world CTR is noisy and expensive to get in real-time, training the GeneAgent using a LightGBM-based surrogate reward model proved highly effective for aligning the agent with commercial goals.
Key Results
CutAgent was tested in a massive one-week online experiment involving over 10 million ad impressions.
- Highest CTR: It achieved an online CTR of 2.21%, significantly outperforming baselines like AutoCut (1.98%) and FireRed-OpenStoryline (1.82%).
- Human Preference: In a forced-choice A/B study, humans preferred CutAgent-generated videos 60.4% of the time.
- Ablation Success: The "Self-Evolve" components (Reflection + Memory) provided consistent gains, raising the CTR from 1.76% (SFT only) to the final 2.21%.
Limitations and Future Work
- Material Dependency: Currently, VideoAgent relies on retrieving clips from a pre-existing pool. The authors suggest that integrating generative video models (like Sora or Gen-3) could fill gaps when specific materials are missing.
- Text Rendering: While the system handles subtitles well, the authors note that more expressive on-screen typography and spatial reasoning for text placement remain areas for improvement.
- Broader Formats: The current focus is on short-form e-commerce videos; extending this "gene-driven" approach to long-form content or different advertising platforms is a logical next step.
Summary updated