Metrics for quantifying isotropy in high dimensional unsupervised clustering tasks in a materials context

Samantha Durdy, Michael W. Gaultois, Vladimir V. Gusev, Danushka Bollegala, Matthew J. RosseinskyarXiv 2023 · DOI · arXiv

Researchers at the University of Liverpool have introduced a new framework for quantifying the "shape" of data clusters in high-dimensional spaces. Traditionally, evaluating cluster geometry has relied on subjective visual inspection of low-dimensional projections. By adapting metrics from medical imaging and natural language processing, this work provides a rigorous mathematical approach to measuring isotropy—the degree to which a cluster is "round" versus "spiky." This quantification is critical for understanding how different data representations and preprocessing techniques influence machine learning models in materials science.

The Challenge of High-Dimensional Shape

In materials science, unsupervised clustering is frequently used to identify groups of similar crystals or molecules. While standard metrics like the Silhouette score measure cluster compactness and separation, they fail to describe the internal geometry of the clusters.

Understanding whether a cluster is a spherical ball (isotropic) or a needle-like structure (anisotropic) is vital. "Spiky" clusters often indicate that a few specific features are dominating the data or that the chosen representation is biased. While Principal Component Analysis (PCA) is often used to visualize these shapes in 2D or 3D, it is inherently subjective and discards significant information. This research provides a way to quantify shape across all dimensions simultaneously.

Methodology: From Brain Imaging to Random Vectors

The study evaluates and extends three primary metrics to capture these geometric nuances:

  1. Fractional Anisotropy (FA): Originally used in medical diffusion tensor imaging to map brain connectivity, this metric measures the variance of the eigenvalues of a cluster's covariance matrix.
  2. Isotropy ($I_c$): Borrowed from word embedding analysis, this measures the ratio of the minimum to maximum cosine similarity between a cluster and a set of vectors.
  3. $I_{c,rnd}$ (The Novel Implementation): The authors propose a computationally efficient version of the $I_c$ metric. Instead of calculating expensive eigenvectors ($I_{c,vec}$), they utilize a set of random unit vectors. They demonstrate mathematically that as the number of random vectors increases, this approximation converges to the true value, offering significant speedups in high-dimensional contexts.
Comparison of isotropy metrics ($I_{c,vec}$ and $I_{c,rnd}$) across different synthetic cluster shapes.

As shown in the figure above, the $I_{c,rnd}$ metric closely tracks the eigenvector-based $I_{c,vec}$ across various geometries, including round (b), elongated (c), and non-convex (a, d) shapes. This validates the use of random vector approximations for rapid shape quantification.

Key Insights: The Sparsity Trap and Generative Quality

The application of these metrics to real-world and synthetic datasets yielded several non-intuitive findings:

  • The Sparsity Trap: Applying a common Kernel Approximation (RBF) had divergent effects based on the initial data representation. It increased the isotropy of dense physical property encodings (magpie) but decreased it for sparse chemical proportion encodings (compVec). This suggests that standard preprocessing steps do not universally "clean" data and can introduce unexpected biases.
  • The Curse of Dimensionality for FA: Using Random Matrix Theory, the authors proved that in very high dimensions, even random noise can appear anisotropic to the FA metric. Consequently, $I_c$ metrics are generally more robust for high-dimensional, noisy datasets.
  • Generative Quality: Analysis of the MNIST dataset revealed that Variational Autoencoders (VAEs) produce significantly more isotropic clusters than standard Autoencoders. This provides a quantitative link between the "roundness" of a latent space and a model's ability to generate realistic new data points.

Conclusion

By moving beyond simple measures of cluster tightness to a formal quantification of shape, this framework allows researchers to better evaluate the impact of data representations. The new $I_{c,rnd}$ method is orders of magnitude faster than traditional approaches in high dimensions ($>1,000$ dimensions), making it a practical tool for large-scale materials discovery. The authors suggest that these metrics could eventually be integrated directly into neural network loss functions to encourage the learning of more useful, isotropic representations.

Summary updated

Abstract

Clustering is a common task in machine learning, but clusters of unlabelled data can be hard to quantify. The application of clustering algorithms in chemistry is often dependant on material representation. Ascertaining the effects of different representations, clustering algorithms, or data transformations on the resulting clusters is difficult due to the dimensionality of these data. We present a thorough analysis of measures for isotropy of a cluster, including a novel implantation based on an existing derivation. Using fractional anisotropy, a common method used in medical imaging for comparison, we then expand these measures to examine the average isotropy of a set of clusters. A use case for such measures is demonstrated by quantifying the effects of kernel approximation functions on different representations of the Inorganic Crystal Structure Database. Broader applicability of these methods is demonstrated in analysing learnt embedding of the MNIST dataset. Random clusters are explored to examine the differences between isotropy measures presented, and to see how each method scales with the dimensionality. Python implementations of these measures are provided for use by the community.
BibTeX
@misc{durdy2023metrics,
  author = {Samantha Durdy and Michael W. Gaultois and Vladimir V. Gusev and Danushka Bollegala and Matthew J. Rosseinsky},
  title = {{Metrics for quantifying isotropy in high dimensional unsupervised clustering tasks in a materials context}},
  year = {2023},
  eprint = {2305.16372},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2305.16372},
}
Figure