Layered Performance Animation with Correlation Maps

Michael Neff, Irene Albrecht, Hans-Peter SeidelComputer Graphics Forum 2007 · DOI

TL;DR

This paper introduces a novel system for "performance animation" that allows animators to control complex 3D humanoid characters using only a simple 2D mouse and keyboard. By employing "correlation maps," the system translates low-dimensional input into expressive, high-dimensional character movements, enabling intuitive and spontaneous animation creation.

What Problem Does This Paper Address?

Traditional animation methods like keyframing offer control but can be laborious and lack the spontaneity of live performance. Straight-ahead animation captures this liveliness but is computationally challenging to implement for complex characters. Existing performance animation systems often require expensive hardware or are limited to low-degree-of-freedom (DOF) characters. This paper addresses the gap of enabling expressive, high-DOF character animation using accessible, low-DOF input devices, bridging the divide between controlled keyframing and spontaneous performance.

Methods

The core innovation is the correlation map. These maps define relationships between 2D mouse input and a set of expressively relevant character parameters. Crucially, these maps can encode two types of correlations:

  • Input-to-character space correlation: How mouse movements directly influence character pose (e.g., moving the mouse down causes the character to hunch).
  • Within-character space correlation: How different character parameters are linked (e.g., spine rotation also causes collar bones to rotate and knees to bend).

The system allows for multiple correlation maps to be active simultaneously, enabling complex movements by combining simpler mappings. Animators can rapidly switch between these maps, effectively changing the character's "rig" on the fly. The system also supports layered animation, where motion is built up over multiple passes, and overlays, which act as dynamic character filters to modify existing motion based on recorded input. The parameterization of character pose itself is also a key contribution, using a set of 33 DOFs derived from artistic research, focusing on expressively salient parameters.

Datasets

Not discussed in the paper. The system is demonstrated with user-created animations and a user study, but no specific datasets are mentioned for training or evaluation.

Key Insights and Ideas

  • Correlation maps are powerful abstractions: By defining relationships rather than direct mappings, animators can achieve complex, nuanced movements from simple inputs. This is a significant departure from literal input-to-DOF mappings.
  • Rapid switching is key to low-DOF control: The ability to quickly swap between different correlation maps is what allows a 2D input to control a high-DOF character. It's like having a dynamic control rig that changes based on the animator's intent.
  • Layering and overlays enable iterative refinement and stylistic control: Building animation in layers and using overlays for stylistic changes offers a flexible way to add detail and explore different character expressions without starting from scratch.
  • Expressively salient parameters are crucial: Using a carefully chosen set of character parameters (like spine curves, balance adjustments, etc.) rather than raw joint DOFs makes the control more intuitive and the resulting animations more natural.

Key Results

  • A user study with novices showed they could create reasonable gesture animations within their first hour of using the system, including training time.
  • The system was used to create a range of animations, including gesturing, dancing, and walking, demonstrating its versatility.
  • Qualitative feedback from the user study indicated that the system encourages creativity and is preferred for sketching initial performances, though some users found traditional tools better for fine-tuning.

Limitations and Future Work

  • Precise editing of recorded values is challenging: While layering and overlays offer flexibility, fine-grained editing of specific moments in recorded motion is difficult. The authors suggest "replace and blend" editing as a potential future extension.
  • Synchronizing with prerecorded audio is difficult: Predicting exact alignment points for audio synchronization remains a challenge.
  • Filtering of mouse input: The authors suggest adding filters to smooth out jerky mouse movements.
  • "Replace and blend" editing: This would allow for re-performing short sections of motion and blending them with existing data.
  • Preview techniques: Enhancing preview capabilities, such as "ghost previews," could aid in synchronization and editing.

Summary updated

BibTeX
@article{neff2007layered,
  author = {Michael Neff and Irene Albrecht and Hans-Peter Seidel},
  title = {{Layered Performance Animation with Correlation Maps}},
  journal = {Computer Graphics Forum},
  year = {2007},
  doi = {10.1111/j.1467-8659.2007.01091.x},
}
Figure