TL;DR
CodeReel is a training-free execution layer that accelerates the "build-inspect-edit" loop for code-to-video agents by over 36%. It achieves this through exact frame reuse, which caches frames based on drawing commands rather than timestamps, and blocking-aware scheduling, which prioritizes the immediate feedback agents need to continue their work.
What Problem Does This Paper Address?
LLM-based coding agents (like Claude Code) generate explainer videos by writing and refining scripts in frameworks like Manim or Remotion. This process is notoriously slow: a single finished film can take 11 minutes to produce, with agents spending more than a third of that time blocked, waiting for "builds" to finish.
The authors identify two primary bottlenecks:
- Redundant Rendering: Even minor edits—like changing a single narration line—often cause the entire video to be re-rendered. Existing caching mechanisms are often too coarse (caching whole scenes) or sensitive to time shifts (caching by frame index), meaning they miss the 37.5% of frames that are actually identical but have simply moved in time.
- Scheduling Mismatch: Current pipelines often render the entire film before performing final checks. However, over half of all builds fail a check (e.g., text overlapping a subject). Agents are forced to wait for a full render just to receive a "fail" verdict they could have gotten in seconds.
Methods
CodeReel introduces two complementary techniques to eliminate this avoidable wait time:
1. Exact Frame Reuse
Instead of indexing frames by their position in time, CodeReel keys each frame by the specific drawing commands (e.g., fillRect, fillText) used to create it.
- Shift-Invariance: If a narration edit moves a scene 0.2 seconds later, the drawing commands remain the same. CodeReel recognizes this and reuses the previously rendered frame from its "frame store."
- Normalization: It applies normalization rules to ensure that minor, non-visual changes (like sub-pixel text positioning) don't trigger unnecessary re-renders.
2. Blocking-Aware Scheduling
CodeReel reorders the work inside a build based on what the agent is currently blocked on:
- Verdict First: It runs linting and layout checks first and in parallel. If a check fails, the agent gets the error message immediately without waiting for any rendering.
- Render as a Bet: It starts rendering alongside checks at idle priority. This "bet" pays off if the build passes, as the film is partially ready. If the build fails, the idle priority ensures the render didn't significantly delay the error verdict.
- Work Ahead: While the agent is "thinking" (processing errors from a failed build), CodeReel continues rendering the rest of that failed version's frames in the background. Because of exact frame reuse, these frames are likely to be useful for the agent's next attempt.
Datasets
The authors introduce EditLoop, a benchmark consisting of 123 real-world recorded sessions of agents (Claude Code and Codex CLI) creating videos.
- It captures every program version, tool call, and the "think time" of the agent.
- It includes two tracks: a
cv.js(Canvas 2D) track and aRemotion(React-based) track. - Unlike previous benchmarks that only grade the final film, EditLoop allows researchers to replay the entire development process to measure system latency and cost.
Key Insights and Ideas
- Intent over Result: By caching the intent to draw (commands) rather than the result (pixels) or context (time), CodeReel achieves near-optimal reuse. On time-shifting edits, it renders only 43.9% of frames, nearly matching the 43.0% of frames that actually contain new pixels.
- Exploiting Agent Idle Time: LLM agents have significant "think time" between builds. CodeReel treats this as a resource, using it to pre-populate the frame cache for future versions.
- Training-Free Efficiency: Unlike many recent AI acceleration methods, CodeReel requires no model training or fine-tuning; it is a pure systems-level optimization that sits behind existing agent commands.
Key Results
CodeReel significantly outperforms existing pipelines on the EditLoop benchmark:
- Wait Time: Reduces the agent's wait by 36.3% compared to a modern parallel pipeline ("Rebuild") and by 57.9% compared to standard sequential pipelines.
- Failure Feedback: It delivers verdicts for failing builds in just 0.39x the time of the baseline.
- Real-World Impact: In live sessions with Claude Code, the median wait per build dropped to 0.62x of the baseline.
- Framework Agnostic: The gains were consistent across five different LLMs and two different video frameworks.
Limitations and Future Work
- 2D Focus: The current implementation is optimized for 2D video programs where drawing commands are easily intercepted.
- 3D and Generative Content: Reusing frames from 3D engines or generative video models (like Sora) is more complex, as their "commands" are not as easily hashed or normalized. The authors suggest this as a primary direction for future research.
- Agent Awareness: Future agents could be made "reuse-aware," planning their edits specifically to maximize the number of frames that can be pulled from the cache.
Summary updated