Designing with Cinematic Language: An Intent-Driven Approach to Human-AI Video Clip Co-Creation

Yi He, Zhengxi Yu, Feilin Han, Chuer Chen, Shengqi Dang, Nan CaoarXiv 2026 · arXiv

TL;DR

CineAssist is a human-AI co-creation system that bridges the "semantic gap" in generative video by translating high-level narrative intents (e.g., "make the character feel isolated") into precise, executable cinematic controls. By grounding the creative process in a structured Cinematic Language Design Space, the system allows users to move beyond trial-and-error prompting toward a professional, multi-agent workflow that includes intent clarification, strategy recommendation, and interactive 3D editing.

What Problem Does This Paper Address?

While generative AI models like Sora or Kling have made video creation accessible, they suffer from a lack of cinematic control. Creators often know what they want to communicate (an emotion or a narrative beat) but struggle to specify how to film it using technical parameters like focal length, camera trajectory, or lighting.

Current tools typically rely on two extremes: "black-box" prompt engineering, which is stochastic and frustrating, or low-level parameter sliders, which require professional filmmaking expertise. The authors identify three core challenges:

  1. Multidimensionality: Cinematic parameters (composition, movement, rhythm) are interdependent.
  2. The Semantic Gap: Users think in high-level intents, while models require concrete coordinates.
  3. Iterative Personalization: Creative styles are personal and evolve, but most AI workflows treat every generation as a fresh start.

Methods

The researchers developed CineAssist, a system built on a multi-agent architecture that treats cinematic language as an intermediate, editable design medium.

  • Cinematic Language Design Space: The heart of the system is a structured mapping of 12 primary intent families (e.g., Emotion Elicitation, Gaze Guidance) and 53 secondary intents to specific cinematic parameter values.
  • Multi-Agent Workflow:
    • Intent Perceiver: An agent that engages the user in a "clarification dialogue" to turn vague requests into a confirmed Intent Brief.
  • Strategy Recommender: Translates the brief into multiple "Shot Scripts"—alternative cinematic strategies that the user can compare.
  • Intent-Oriented Editor: A 3D workspace where users can directly manipulate the camera and subjects. Crucially, the editor provides "intent-aware" guidance, showing how moving the camera might strengthen or weaken the intended emotional effect.
  • Preference Learner: A context-sensitive module that observes user edits over time. It doesn't just learn that a user likes "wide shots," but specifically that they prefer "wide shots for conveying loneliness," allowing for nuanced personalization in future tasks.
Figure 1: Overview of CineAssist. Guided by (a) the cinematic language design space, (b) the Intent Perceiver clarifies vague requests into explicit intentions, and (c) the Strategy Recommender translates them into executable Shot Scripts. (d) The editor instantiates and refines the resulting camera configurations in 3D, while (e) the Preference Learner learns reusable preferences from creators’ choices and edits. Together, these modules produce controllable cinematic rewrites that communicate the intended effect.

Datasets

To ground the system in real-world practice, the authors constructed a new, large-scale cinematic dataset:

  • Sources: 107 filmmaking textbooks and 1,436 film clips sampled from Academy Award nominees, the IMDb Top 250, and professional movie clip archives.
  • Annotation: The team used an AI-assisted pipeline (verified by human experts) to produce 3,895 labels in the form of <intent, cinematic-parameter-values>. This dataset provides the empirical evidence for which camera angles or lighting setups actually correlate with specific narrative goals.

Key Insights and Ideas

  • Cinematic Language as an Intermediate Representation: The most significant "aha moment" is that we shouldn't go directly from text to video. By inserting a "Cinematic Language" layer, the AI can explain why it chose a specific camera move, and the human can edit that reasoning directly.
  • Decision-Centered vs. Generation-Centered: The system shifts the AI's role from an autonomous "creator" to a "creative partner." The AI structures the difficult design space and proposes alternatives, but the human retains authorship through the 3D editor.
  • Contextual Personalization: The Preference Learner uses an "evidence-based progression" (moving from Observed to Corroborated). This prevents a single outlier choice from permanently skewing the AI's understanding of a user's style.

Key Results

The authors evaluated CineAssist through a user study with 30 participants and a case study with 6 experts (directors and AI practitioners).

  • Quantitative Performance: Compared to a state-of-the-art baseline (Seedance 2.0), videos generated with CineAssist were rated significantly higher ($p < .001$) for clarity of intent, cinematic appropriateness, and overall preference.
  • Preference Adaptation: The Preference Learner achieved 77.8% accuracy in transferring learned styles to new visual contexts.
  • Qualitative Feedback: Experts noted that the 3D editor felt like "arranging a shot on set," and AIGC practitioners found the intent-driven guidance much more intuitive than "spending an entire afternoon generating variations" via prompts.
Qualitative comparison showing how CineAssist produces more intent-aligned results compared to direct generation.

Limitations and Future Work

  • Short-Form Focus: The current system is optimized for single shots or short sequences. The authors acknowledge that extending this to long-form narratives—handling continuity and pacing across multiple scenes—is a major next step.
  • Situated Knowledge: Cinematic techniques are not universal; they vary by genre and culture. Future versions could incorporate "genre-specific" design spaces (e.g., a "Horror" vs. "Comedy" mode).
  • Professional Integration: While CineAssist is a standalone tool, the authors suggest it could eventually serve as an "intent-aware layer" within professional suites like Premiere Pro or DaVinci Resolve.

Key Results

Summarize the main quantitative results (accuracy, speedup, etc.) and qualitative findings. Use concrete numbers when available.

Limitations and Future Work

Honestly note limitations the authors acknowledge (or that are apparent) and promising directions they suggest.

Summary updated

Abstract

Generative video models have made video creation more accessible, yet creators still struggle to translate high-level narrative intentions into coherent cinematic expressions. Existing systems primarily rely on prompt refinement or expose isolated camera controls, leaving a gap between what to express and how to film it. We present CineAssist, a human-AI co-creation system that treats cinematic language as an interactive design medium connecting creative intentions with executable video controls. We construct a Cinematic Language Design Space and an annotated dataset from 107 filmmaking sources and 1,436 film clips, linking communicative intentions to cinematic parameter values. CineAssist combines a multi-agent workflow for intent alignment and strategy recommendation with an interactive cinematic editor and a context-sensitive preference learner for user-controlled personalization. We evaluate the system through three studies examining end-to-end usability, cinematic quality, and preference evolution. Results show that CineAssist facilitates efficient, controllable, and personalized intent-driven cinematic video creation.
BibTeX
@misc{he2026designing,
  author = {Yi He and Zhengxi Yu and Feilin Han and Chuer Chen and Shengqi Dang and Nan Cao},
  title = {{Designing with Cinematic Language: An Intent-Driven Approach to Human-AI Video Clip Co-Creation}},
  year = {2026},
  eprint = {2611.07377},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.07377},
}
Figure