Tree-of-AdEditor: Heuristic Tree Reasoning for Automated Video Advertisement Editing with Large Language Model

Yuqi Zhang, Bin Guo, Nuo Li, Ying Zhang, Shijie Wang, Zhiwen Yu, Qing LiIJCAI 2025 · DOI · OpenReview

TL;DR

This paper introduces Tree-of-AdEditor (ToAE), a novel framework that uses Large Language Models (LLMs) to automate video advertisement editing. ToAE mimics human editors by employing a tree-based reasoning process, incorporating domain-specific advertising principles, and using visual heuristics for fact-checking, outperforming existing methods in generating more coherent and attractive ads.

What Problem Does This Paper Address?

Creating effective video advertisements for e-commerce is challenging. It requires not only showcasing product selling points and maintaining a coherent narrative but also aligning with target audience preferences. Traditional methods often rely on rigid rules or require massive datasets and high training costs. While LLMs show promise, existing prompting techniques like Input-Output (IO) and Chain-of-Thought (CoT) struggle with the iterative, non-linear nature of video editing. Tree-of-Thoughts (ToT) offers a tree-like structure but lacks domain-specific guidance for advertising and robust fact-checking, leading to potentially suboptimal or visually jarring results. This paper aims to bridge this gap by developing an LLM-based editing system that captures the nuanced decision-making of human editors.

Methods

The core of ToAE is a heuristic tree reasoning framework designed to mimic human editors. Key components include:

  • Explicit Shot Semantic Representation: Each video clip is converted into a textual description capturing its content (main object, action, background) and cinematographic elements (shot scale, camera angle). This bridges the gap between visual data and LLM understanding.
  • Local-Global Guidance: This mechanism guides the LLM's decision-making at two levels:
  • Local Next Shot Selector: Recommends the next best shot based on conveying selling points, narrative structure, and visual diversity.
  • Global Sequence Evaluator: Assesses entire editing plans for stylistic alignment with the target audience, overall coherence, attractiveness, and conveyance of selling points.
  • Visual Incoherence Pruning: This module acts as an external fact-checker, identifying and pruning branches of the reasoning tree that lead to visually abrupt or incoherent transitions (e.g., jump cuts, intensity differences). This improves visual appeal and reduces computational cost.

The framework uses a depth-first search (DFS) to explore potential editing paths, pruning branches that violate visual coherence rules, and finally selecting the plan with the highest score from the global evaluator.

Datasets

The paper evaluates ToAE on two datasets:

  • MovingFashion-AVE: A dataset of fashion video advertisements from Net-A-Porter.
  • ProductAVE: A newly introduced dataset comprising 152 Taobao video advertisements across various e-commerce categories.

Key Insights and Ideas

  • Mimicking Human Iteration is Crucial: The paper highlights that video editing isn't a linear process. ToAE's tree-based approach, with its "select" and "put back" capabilities, better reflects how human editors explore options.
  • Domain-Specific Guidance Matters: Simply using LLMs for general reasoning isn't enough. ToAE's success stems from incorporating principles of effective advertising (informativeness, coherence, attractiveness) and target audience preferences into its guidance mechanisms.
  • Visual Fact-Checking is Essential: Relying solely on textual descriptions can lead to visually unappealing ads. The visual incoherence pruning module is a critical addition that significantly improves the aesthetic quality and efficiency of the editing process.
  • Local and Global Perspectives Enhance Decision-Making: The combination of local shot selection and global sequence evaluation allows ToAE to make both fine-grained and holistic decisions, leading to more well-rounded advertisements.

Key Results

  • Superior Performance: ToAE consistently outperformed baselines (IO, CoT, ToT, and learning-based methods like IIC and LMP) across both datasets on key metrics like Conveyance, Coherence, Attractiveness, and Stylistic Alignment.
  • Significant Visual Coherence Improvement: ToAE reduced visual coherence errors (Jump cut, Opposite Motion, Intensity Difference, Tonal Mismatch) by substantial margins compared to baselines. For instance, it reduced Tonal Mismatch by at least 10% and Opposite Camera Motion by at least 15%.
  • Human Evaluation Validation: In a human evaluation with industry professionals, ToAE was favored by 43% of participants, significantly higher than CoT (25%) and ToT (32%). Professionals rated ToAE's generated ads higher across all dimensions.
  • Efficiency Gains: ToAE significantly reduces inference steps compared to naive enumeration, making it computationally feasible for practical use.

Limitations and Future Work

  • Limited Scope: The current framework focuses on shot selection and assembly. Future work could integrate music and special effects to create more complete advertisements.
  • Longer Ads: While ToAE shows effectiveness for typical-length ads, challenges remain for producing very long advertisements.
  • LLM Foundation Variability: While ToAE demonstrates robustness across different LLM foundations (Mistral, GPT-40-mini), further exploration into optimizing prompts for specific models could be beneficial.

Summary updated

Abstract

Video advertising has become a popular marketing strategy on e-commerce platforms, requiring high-level semantic reasoning like selling point discovery, narrative organization. Previous rule-based methods struggle with these complex tasks, and learning-based approaches demand large datasets and high training costs. Recently, Large Language Models have opened incredible opportunities for advancing intelligent video advertisement editing. However, Input-output (IO) prompting and Chain-of-Thought (CoT) struggle to adapt to the nonlinear thinking hierarchy of video editing, where editors iteratively select shots or revert them to explore potential editing solutions. While Tree-of-Thought (ToT) offers a conceptual structure that mirrors this hierarchy, it falls short in aligning with effective video advertising strategies and lacks robust fact-checking mechanisms. To address these, we propose a novel framework, Tree-of-AdEditor (ToAE), which constructs a reasoning tree to mimic human editors, and incorporates domain-specific theories and heuristic fact-checking to identify optimal editing solutions. Specifically, motivated by effective advertisement principles, we develop a "local-global" mechanism to guide LLM in both the shot level and sequence level decision-making. We introduce a visual incoherence pruning module to provide external heuristic fact-checking, ensuring visual attractiveness and reducing computation costs. Quantitative experiments and expert evaluation demonstrate the superiority of our method compared to baselines.
BibTeX
@inproceedings{zhang2025tree,
  author = {Yuqi Zhang and Bin Guo and Nuo Li and Ying Zhang and Shijie Wang and Zhiwen Yu and Qing Li},
  title = {{Tree-of-AdEditor: Heuristic Tree Reasoning for Automated Video Advertisement Editing with Large Language Model}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.24963/ijcai.2025/968},
}
Figure