Infinite ShapeOdds: Nonparametric Bayesian Models for Shape Representations

Wei W. Xing, Shireen Y. Elhabian, Robert Michael Kirby, Ross T. Whitaker, Shandian Zhe

Introduction

In fields such as medical imaging and computer vision, representing complex binary shapes (e.g., organ silhouettes or object outlines) using compact, low-dimensional latent codes is a fundamental challenge. While deep learning models like Variational Auto-encoders (VAEs) have become the standard for such tasks, they often struggle in specialized domains where data is scarce. VAEs are notoriously data-hungry and prone to overfitting on small datasets. Furthermore, traditional robust models like ShapeOdds rely on linear projections, which fail to capture the intrinsic nonlinear variations of natural shapes, such as the bending of a limb or the rotation of an object.

To address these limitations, we present Infinite ShapeOdds (InfShapeOdds), a Bayesian nonparametric framework designed to learn flexible, low-dimensional representations of binary shapes. By integrating Gaussian Processes (GPs) with Dirichlet Process Mixtures (DPMs), InfShapeOdds automatically discovers hidden clusters and captures nonlinear variations without the extensive hyperparameter tuning required by deep learning approaches.

Methodology

InfShapeOdds is built upon three core mathematical innovations:

  1. Shape-Variate Gaussian Processes (GP): Unlike linear models that use simple projections, InfShapeOdds employs a GP to map latent codes to pixel space. This allows the model to capture complex, nonlinear correlations between pixels. To manage the computational complexity of high-dimensional images, the authors utilize a Kronecker-product structure ($K_S \otimes K_T$) for the spatial kernel. This decomposes spatial correlations into horizontal and vertical components, reducing the computational cost from $O(D^3)$ to $O(I^3 + J^3)$ for an $I \times J$ image.
  2. Dirichlet Process Mixture (DPM) Prior: The "Infinite" component of the model refers to the DPM prior placed over the latent space. This allows the model to infer the number of distinct clusters (e.g., different poses of an animal) directly from the data, eliminating the need to pre-specify the number of clusters as in K-means.
  3. Efficient Variational Inference: The model utilizes a truncated variational expectation-maximization (EM) algorithm. This approach enables efficient estimation of parameters and latent representations while maintaining the benefits of the Bayesian framework.

Key Results and Discussion

Robustness to Overfitting

A significant advantage of the Bayesian nonparametric approach is its inherent robustness. In comparative tests against VAEs, InfShapeOdds demonstrated superior stability. While VAEs often show a diverging gap between training and test error as training progresses—indicating overfitting—the test and training errors for InfShapeOdds converge and remain stable.

Figure 1: Cross-entropy error over training epochs. Unlike VAEs, InfShapeOdds (red) shows stable test performance without overfitting as training continues.

Visual Fidelity and Reconstruction

Quantitatively, InfShapeOdds consistently achieves lower cross-entropy errors than VAEs, GPLVM, and the original linear ShapeOdds across various training sample ratios (35%, 55%, and 75%). Qualitatively, the difference is even more pronounced. Linear models often produce "ghostly" or blurry averages when representing variations. In contrast, InfShapeOdds produces sharp, realistic reconstructions and interpolations by accurately modeling the nonlinear manifold of the shape data.

Figure 5: Visual comparison of shape reconstructions for the Horse and Motorcycle datasets. InfShapeOdds maintains sharper edges and better structural integrity compared to competing methods.

Clustering and Latent Space

On the Fashion-MNIST dataset, InfShapeOdds achieved higher clustering purity scores than baseline models. By jointly learning the latent representation and the cluster assignments through the DPM prior, the model identifies meaningful sub-categories of objects without supervision. This tight coupling ensures that the low-dimensional space is optimized for both reconstruction and categorization.

Conclusion and Future Work

InfShapeOdds provides a powerful alternative to deep learning for shape modeling, particularly when data is limited. By leveraging the flexibility of GPs and the automated complexity of DPMs, it captures nonlinear shape variations more effectively than linear methods while avoiding the overfitting pitfalls of VAEs.

Future research will focus on scaling the Kronecker-product GP to higher-resolution images using inducing-point methods and extending the framework to handle multi-channel (RGB) or continuous-valued image data by adapting the likelihood function.

Summary updated

Abstract

Learning compact representations for shapes (binary images) is important for many applications. Although neural network models are very powerful, they usually involve many parameters, require substantial tuning efforts and easily overfit small datasets, which are common in shape-related applications. The state-of-the-art approach, ShapeOdds, as a latent Gaussian model, can effectively prevent overfitting and is more robust. Nonetheless, it relies on a linear projection assumption and is incapable of capturing intrinsic nonlinear shape variations, hence may leading to inferior representations and structure discovery. To address these issues, we propose Infinite ShapeOdds (InfShapeOdds), a Bayesian nonparametric shape model, which is flexible enough to capture complex shape variations and discover hidden cluster structures, while still avoiding overfitting. Specifically, we use matrix Gaussian priors, nonlinear feature mappings and the kernel trick to generalize ShapeOdds to a shape-variate Gaussian process model, which can grasp various nonlinear correlations among the pixels within and across (different) shapes. To further discover the hidden structures in data, we place a Dirichlet process mixture (DPM) prior over the representations to jointly infer the cluster number and memberships. Finally, we exploit the Kronecker-product structure in our model to develop an efficient, truncated variational expectation-maximization algorithm for model estimation. On synthetic and real-world data, we show the advantage of our method in both representation learning and latent structure discovery.
BibTeX
@inproceedings{xing2020infinite,
  author = {Wei W. Xing and Shireen Y. Elhabian and Robert Michael Kirby and Ross T. Whitaker and Shandian Zhe},
  title = {{Infinite ShapeOdds: Nonparametric Bayesian Models for Shape Representations}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.1609/aaai.v34i04.6118},
}
Figure