TL;DR
AnimWeb-Bench is the first benchmark designed to evaluate how well AI models can generate functional webpage code from a reference video of an animated site. While current Multimodal Large Language Models (MLLMs) are becoming adept at recreating static layouts, this research reveals they still struggle significantly with the continuous temporal dynamics—timing, easing, and trajectories—that define the modern web.
What Problem Does This Paper Address?
The "screenshot-to-code" pipeline has seen massive improvements recently, with models like GPT-4o and Claude 3.5 Sonnet capable of turning a static image into a decent HTML/CSS layout. However, the web isn't static. Over 75% of webpages utilize CSS animations to provide feedback, direct attention, or simply enhance aesthetics.
Existing benchmarks typically focus on static screenshots or discrete, user-triggered interactions (like clicking a button). They ignore the continuous dynamics of time-driven animations that unfold automatically during page load. Reconstructing these requires a model to not only understand what is on the page but how and when every element moves. AnimWeb-Bench fills this gap by challenging models to generate a single, executable HTML file (including inline CSS and JavaScript) that reproduces a reference video's visual and motion characteristics.
Methods
The authors developed a comprehensive pipeline for both task definition and evaluation:
- Task Definition: Given a reference video and a set of static assets (images, icons), a model must generate code. This code is then rendered in a browser to produce a "generated video" for comparison.
- Keyframe Alignment: To avoid the redundancy of comparing every single frame, the system automatically detects "stable states" (keyframes) and "animation phases" (segments between keyframes) using a visual similarity curve.
- Two-Dimensional Evaluation Protocol:
- Static Appearance Fidelity: Measures how well the generated keyframes match the reference in terms of structure (box overlap), text (OCR), color, and high-level visual similarity (CLIP).
- Animation Fidelity: This is the novel core. It tracks UI components across frames to compare their Position, Size, Velocity, and Appearance (CLIP) against the reference trajectories.
Datasets
The authors introduced AnimWeb-Bench, which consists of:
- 124 real-world webpages sourced from Awwwards, a showcase for high-end web design.
- 7 application domains (Technology, Media, Design, etc.) and 33 distinct animation types (e.g., translation, masking, text scramble).
- High Complexity: Unlike previous "motion graphics" datasets that used simple snippets, AnimWeb-Bench pages average nearly 1,200 HTML tags, compared to just ~70 in earlier benchmarks like Animation2Code.
Key Insights and Ideas
- "Systems Look Right but Move Wrong": Models often achieve respectable scores on static appearance but fail miserably on animation fidelity. Velocity is consistently the lowest-scoring metric, indicating that models struggle to capture the "feel" and speed of motion.
- Perception is the Bottleneck: Coding agents (models that can use tools) perform better when they are explicitly told to inspect high-resolution frames individually rather than looking at low-res "contact sheets" of video frames.
- The Power of the Loop: Providing "Render Feedback"—allowing an agent to record its own generated code and compare it to the reference—significantly improves animation quality. It allows the agent to "see" its timing errors and iterate.
- Continuous vs. Discrete: Rapid, continuous changes (like a "Text Scramble" effect) are much harder for AI to reproduce than discrete state transitions (like a simple "Fade In"). The former requires precise frame-by-frame synchronization that current models cannot yet express in code.
Key Results
The study evaluated 14 MLLMs and 3 coding agents.
- Top Performer: GPT-6 Astra (a placeholder name for the state-of-the-art model at the time of writing) led the pack, but even it scored below 0.46 on all animation metrics.
- The "Velocity" Gap: While static CLIP scores reached ~0.88 for top models, Velocity scores hovered around 0.30, highlighting the difficulty of temporal reconstruction.
- Agentic Gains: A coding agent (Codex) using Frame Inspection (FI) and Render Feedback (RF) saw relative gains of up to 22% in static metrics and 15% in animation metrics compared to a single-pass approach.
- Human Alignment: The automated metrics showed a strong correlation (Spearman’s $\rho = 0.67–0.74$) with 400 pairwise human judgments, validating the benchmark's utility.
Limitations and Future Work
The authors acknowledge several areas for growth:
- Low Absolute Performance: Even the best models are far from "perfect" reconstruction, suggesting that temporal visual understanding is still a major frontier for MLLMs.
- Interaction Complexity: While this benchmark focuses on time-driven animations, future versions could incorporate complex user-triggered sequences (e.g., drag-and-drop or multi-step forms).
- Code Efficiency: The benchmark evaluates visual output but doesn't deeply penalize "spaghetti code" or inefficient JavaScript, provided the visual result is correct. Future work could integrate code quality metrics.
By providing a rigorous way to measure "motion fidelity," AnimWeb-Bench sets a new bar for the next generation of AI web developers.
Summary updated