Approximating Intersections and Differences Between Linear Statistical Shape Models Using Markov Chain Monte Carlo

Maximilian Weiherer, Finn Klein, Bernhard Egger

Introduction

Statistical Shape Models (SSMs) are fundamental in computer vision and medical imaging for representing complex objects like human faces or organs in low-dimensional subspaces. While these models are powerful for data fitting and synthesis, comparing two different SSMs—for instance, one trained on male faces and another on female faces—has historically been limited to scalar metrics like compactness, generalization, or specificity. These metrics describe performance but fail to reveal the actual geometric or statistical overlap between the shapes the models represent.

The Challenge of Subspace Comparison

The core difficulty lies in the high-dimensional nature of shape data. While a face model might have thousands of vertices, the "allowable" shapes occupy a tiny, low-dimensional subspace. Directly calculating the geometric intersection of these subspaces is mathematically problematic because real-world models are noisy and rarely intersect perfectly. Consequently, there has been a lack of tools to visualize the "shared" space between models or the specific features that distinguish one from another.

Methodology: MCMC for Shape Set Operations

The authors propose a framework that treats shape model comparison as a probabilistic inference task rather than a pure linear algebra problem. Instead of looking at infinite vector spaces, they focus on the "allowable shape domain"—the region where shapes are statistically plausible according to the models.

Conceptual overview of set-theoretic operations on synthetic star models.

The method employs Markov Chain Monte Carlo (MCMC) sampling, specifically the Metropolis-Hastings algorithm, to explore the intersection and difference of two models:

  1. Likelihood Function: To determine if a shape $x$ from Model A belongs in the intersection with Model B, it is projected onto Model B’s subspace. If the projection distance is small and the resulting point is statistically likely within Model B, it is considered part of the intersection.
  2. Intersection SSM: Samples generated from the intersection are used to train a new linear SSM via Principal Component Analysis (PCA). This "Intersection Model" represents the shared features.
  3. Set-Theoretic Difference: To find shapes unique to Model A, the distance likelihood is inverted. Because the resulting space is non-convex (containing a "hole" where the intersection was), PCA cannot be applied. Instead, the authors visualize the raw MCMC samples to understand unique characteristics.

Key Results and Insights

The framework was validated using a synthetic "Star Model" where ground-truth intersections could be precisely controlled. The MCMC method recovered these intersections with high quantitative accuracy, achieving an average Grassmann distance of 0.0194.

MCMC-based approximations of the intersection and differences.

Beyond synthetic data, the method provided significant qualitative insights into real-world models:

  • Gender Neutrality: Intersecting male and female face models from the Liverpool York Head Model (LYHM) naturally produced a gender-neutral "Intersection SSM." The "Difference" samples highlighted hyper-masculine or hyper-feminine traits.
  • Identity-Expression Ambiguity: Using the Basel Face Model (BFM 2019), the authors quantified the overlap between identity and expression models. The method visually pinpointed "leakage," showing how expression models often inadvertently capture identity information.
  • Feature Localization: Visualizations revealed which anatomical parts contribute most to model differences. For example, the nose was identified as a primary feature for identity, while the mouth and cheeks dominated expression-specific models.

Conclusion

By moving beyond scalar performance metrics, this MCMC-based approach provides a deeper qualitative and quantitative understanding of how Statistical Shape Models relate to one another. While challenges remain regarding the "curse of dimensionality" in very high-dimensional intersection spaces, the ability to visualize commonalities and differences offers a powerful new tool for model analysis and decoupling in computer vision.

Summary updated

Abstract

To date, the comparison of Statistical Shape Models (SSMs) is often solely performance-based, carried out by means of simplistic metrics such as compactness, generalization, or specificity. Any similarities or differences between the actual shape spaces can neither be visualized nor quantified. In this paper, we present a new method to qualitatively compare two linear SSMs in dense correspondence by computing approximate intersection spaces and set-theoretic differences between the (hyper-ellipsoidal) allowable shape domains spanned by the models. To this end, we approximate the distribution of shapes lying in the intersection space using Markov chain Monte Carlo and subsequently apply Principal Component Analysis (PCA) to the posterior samples, eventually yielding a new SSM of the intersection space. We estimate differences between linear SSMs in a similar manner; here, however, the resulting spaces are no longer convex and we do not apply PCA but instead use the posterior samples for visualization. We showcase the proposed algorithm qualitatively by computing and analyzing intersection spaces and differences between publicly available face models, focusing on gender-specific male and female as well as identity and expression models. Our quantitative evaluation based on SSMs built from synthetic and real-world data sets provides detailed evidence that the introduced method is able to recover ground-truth intersection spaces and differences accurately.
BibTeX
@inproceedings{weiherer2024approximating,
  author = {Maximilian Weiherer and Finn Klein and Bernhard Egger},
  title = {{Approximating Intersections and Differences Between Linear Statistical Shape Models Using Markov Chain Monte Carlo}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2024},
  doi = {10.1109/wacv57701.2024.00624},
}
Figure