Abstract
We describe a very simple framework for deriving the most-well known optimization problems in Active Appearance Models (AAMs), and most importantly for providing efficient solutions. Our formulation results in two optimization problems for fast and exact AAM fitting, and one new algorithm which has the important advantage of being applicable to 3D. We show that the dominant cost for both forward and inverse algorithms is a few times $mN$ which is the cost of projecting an image onto the appearance subspace. This makes both algorithms not only computationally realizable but also very attractive speed-wise for most current systems. Because exact AAM fitting is no longer computationally prohibitive, we trained AAMs in-the-wild with the goal of investigating whether AAMs benefit from such a training process. Our results show that although we did not use sophisticated shape priors, robust features or robust norms for improving performance, AAMs perform notably well and in some cases comparably with current state-of-the-art methods. We provide Matlab source code for training, fitting and reproducing the results presented in this paper at http://ibug.doc.ic.ac.uk/resources.
Introduction
Active Appearance Models (AAMs) have long been a staple of computer vision for object tracking and alignment. However, their application to "in-the-wild" scenarios—characterized by significant variations in pose, expression, and illumination—was historically hampered by a computational bottleneck. Researchers were forced to choose between the high accuracy but slow performance of the Simultaneous Inverse Compositional (SIC) algorithm and the fast but often inaccurate Project-Out Inverse Compositional (POIC) method.
This paper addresses this gap by introducing a mathematical framework that achieves the exact solution of SIC with the efficiency of POIC, enabling robust AAM fitting in unconstrained environments.
Methodology: Fast AAM Optimization
The core innovation lies in the analytical decoupling of shape and appearance parameters. By utilizing the property that a function of two variables can be minimized by first solving for one analytically, the authors solve for appearance parameters ($c$) by projecting the image onto the appearance subspace. This reduces the iterative optimization problem to the shape parameters ($p$) alone:
$$\min_{x,y} f(x, y) = \min_x [\min_y f(x, y)]$$
Two primary algorithms are derived from this framework:
- Fast-SIC: An exact version of the inverse compositional algorithm. By re-ordering linearization and projection steps, the authors reduce the complexity from $O((n+m)^2N)$ to $O(nmN)$.
- Fast-Forward: A new forward-additive algorithm that is also $O(nmN)$. Unlike inverse methods, this approach can be easily extended to 3D models.
In both cases, the dominant cost becomes simply projecting the image onto the appearance subspace—a step that is essentially the "entry fee" for any appearance-based model.
Experimental Results
The proposed algorithms were evaluated on challenging, unconstrained datasets including Labeled Face Parts in the Wild (LFPW) and Helen. These datasets contain images from the web with massive variations in lighting and pose.
The results demonstrate that Fast-SIC and Fast-Forward significantly outperform the traditional POIC method. Furthermore, the study reveals that AAMs trained on "in-the-wild" data can achieve performance comparable to state-of-the-art methods using only raw pixel intensities, without the need for complex feature descriptors like SIFT or HOG. This suggests that the model's clean representation is often superior to noisy image gradients.
Conclusion
By solving the efficiency problem of exact AAM fitting, this work revitalizes AAMs for modern, unconstrained computer vision tasks. The findings suggest that the simplicity of raw pixels, when combined with exact optimization and representative training data, is surprisingly effective. Future work may involve integrating robust features and exploring approximate Hessians for further speed improvements.
Reference: Tzimiropoulos, G., & Pantic, M. (2013). Optimization problems for fast AAM fitting in-the-wild. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 593-600.
Summary updated