Pairwise residue-residue Euclidean distances for a Histatin-5 ensemble

Javier González-Delgado, Mathis Deronzier, Juan Cortés, Pierre NeuvialarXiv 2023 · DOI · arXiv

TL;DR

This paper generalizes post-clustering inference to data with complex dependencies between observations and features. By extending the matrix normal model, the authors provide a framework to test if cluster means are truly different without "double-dipping" bias, even when dealing with highly correlated data like protein structures.

What Problem Does This Paper Address?

In data science, we often use clustering to find patterns and then perform hypothesis tests to see if the resulting groups are statistically different. However, using the same data for both tasks is a form of "double-dipping" (or data snooping) that invalidates standard p-values, leading to a high rate of false discoveries.

While recent "selective inference" frameworks (notably Gao et al., 2022) have addressed this for independent and identically distributed (i.i.d.) Gaussian data, real-world data is rarely that simple. In fields like structural biology or econometrics, observations are often dependent (e.g., time-series) and features are highly correlated (e.g., distances between atoms in a protein). This paper tackles the challenge of performing valid inference when the data follows a Matrix Normal distribution with arbitrary dependence structures.

Methods

The authors extend the existing framework to the general matrix normal model: $X \sim \mathcal{MN}_{n \times p}(\mu, U, \Sigma)$, where $U$ represents the dependence between observations and $\Sigma$ represents the dependence between features.

Key technical contributions include:

  • The $p_{gen}$ p-value: A generalized p-value that accounts for the geometry of the clustering algorithm and the underlying covariance structure.
  • Compound Symmetry Condition: They prove that if the observation dependence $U$ has a "compound symmetry" structure (where every pair of observations has the same correlation), the selective Type I error can be controlled exactly.
  • Covariance Estimation: They provide theoretical conditions under which an unknown feature covariance $\Sigma$ can be estimated. Crucially, they show that asymptotically over-estimating the covariance (using the Loewner partial order) still results in valid, albeit conservative, p-values.
  • Algorithm Support: The theory is developed for Hierarchical Agglomerative Clustering (HAC) with various linkages (average, Ward, etc.) and the $k$-means algorithm.

Datasets

  • Synthetic Data: The authors used simulated matrix normal samples with various covariance structures, including AR(1), Toeplitz, and diagonal matrices, to validate error control and power.
  • Protein Conformational Data (Histatin-5): The primary real-world application uses a dataset of 2,000 conformations of the disordered protein Histatin-5. The features are 276 pairwise inter-residue Euclidean distances. This dataset is characterized by high feature correlation and was used to demonstrate how the method prevents over-clustering in structural biology.

Key Insights and Ideas

  • The "Whitening" Trap: A common instinct is to "whiten" the data (decorrelate it) before clustering. The authors demonstrate that this is often a mistake; whitening can fundamentally change the cluster structure, leading to different and potentially less biologically relevant hypotheses.
  • Robustness to Misspecification: One of the most practical "aha" moments is that the proposed test is robust. Even if the observation dependence $U$ doesn't perfectly fit the "compound symmetry" model, the p-values often still maintain statistical guarantees in practice.
  • Safety in Over-estimation: In many cases, we don't know the true covariance. The authors show that it is statistically "safe" to use an estimator that over-estimates the variance, as it ensures the p-values remain super-uniform (conservative) rather than anti-conservative (leading to false positives).

Key Results

  • Error Control: Unlike naive methods that ignore dependence (which produce p-values skewed toward zero), the proposed $p_{gen}$ produces a uniform distribution under the null hypothesis, successfully controlling the selective Type I error.
  • Power Analysis: Among clustering methods, HAC with average linkage consistently showed the highest relative efficiency and the lowest power loss when the covariance matrix had to be estimated.
  • Biological Discovery: In the Histatin-5 analysis, the method revealed that several clusters found by the algorithm were not significantly different. This allowed researchers to merge groups into a more robust subgroup characterized by extended protein structures, providing a clearer picture of the protein's conformational landscape.

Limitations and Future Work

  • Arbitrary Observation Dependence: While the paper proposes a "p-Gamma" approach for arbitrary $U$, proving exact error control for this case remains a significant theoretical challenge.
  • Power Loss from Conditioning: The process of "conditioning" on the clustering event to ensure validity inherently reduces the statistical power of the test. Finding ways to reduce this power loss is an open area of research.
  • Simultaneous Estimation: Estimating both $U$ and $\Sigma$ simultaneously from a single copy of the data is currently unfeasible; the authors suggest that future work could explore alternative test statistics that do not depend on these unknown parameters.

Summary updated

Abstract

Pairwise residue-residue Euclidean distances for a pair of independent replicas of the protein ensemble Histatin-5. Each file corresponds to a n x p matrix, where n is the number of conformers and p is the number of pairwise inter-residue distances, that is, p = L(L-1)/2 where L is the number of sequence residues. Each row corresponds to a p-vector of distances featuring the corresponding conformation. Distances are computed between CB atoms (CA for glycines) and are given in Angstroms (Å). Conformations were generated using Flexible-Meccano (Ozenne et al. 2012, Bernado et al. 2005) and refined using previously reported small-angle X-ray scattering (SAXS) data (Sagar et al. 2021).
BibTeX
@misc{gonzalezdelgado2023pairwise,
  author = {Javier González-Delgado and Mathis Deronzier and Juan Cortés and Pierre Neuvial},
  title = {{Pairwise residue-residue Euclidean distances for a Histatin-5 ensemble}},
  year = {2023},
  eprint = {2310.11822},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2310.11822},
}
Figure