Abstract
Representational similarity analysis (RSA) summarizes activity patterns for a set of experimental conditions into a matrix composed of pairwise comparisons between activity patterns. Two examples of such matrices are the condition-by-condition inner product matrix or the correlation matrix. These representational matrices reside on the manifold of positive semidefinite matrices, called the Riemannian manifold. We hypothesize that representational similarities would be more accurately quantified by considering the underlying manifold of the representational matrices. Thus, we introduce the distance on the Riemannian manifold as a metric for comparing representations. Analyzing simulated and real fMRI data and considering a wide range of metrics, we show that the Riemannian distance is least susceptible to sampling bias, results in larger intra-subject reliability, and affords searchlight mapping with high sensitivity and specificity. Furthermore, we show that the Riemannian distance can be used for measuring multi-dimensional connectivity. This measure captures both univariate and multivariate connectivity and is also more sensitive to nonlinear regional interactions compared to the state-of-the-art measures. Applying our proposed metric to neural network representations of natural images, we demonstrate that it also possesses outstanding performance in quantifying similarity in models. Taken together, our results lend credence to the proposition that RSA should consider the manifold of the representational matrices to summarize response patterns in the brain and models.
The Challenge of Representational Similarity
In systems neuroscience, researchers frequently compare how different brain regions or artificial neural networks represent the world. This is typically achieved through Representational Similarity Analysis (RSA), which summarizes complex activity patterns into "representational matrices" (such as correlation or second-moment matrices).
However, a fundamental mathematical oversight exists in standard RSA: these matrices are Positive Semidefinite (PSD). They do not exist in a standard flat, Euclidean space; instead, they reside on a curved Riemannian manifold. Most current research uses standard metrics like Pearson correlation or Euclidean distance to compare these matrices, effectively "flattening" the manifold and ignoring its natural geometry. This simplification leads to significant issues, including high sensitivity to sampling bias (voxel/neuron count), low reliability across different fMRI runs, and an inability to capture sophisticated, nonlinear relationships between brain regions.
Geometry of the Riemannian Manifold
The core proposition of this work is that representational similarities are more accurately quantified by respecting the underlying geometry of the PSD manifold. Instead of calculating a straight line between two points (Euclidean distance), the Riemannian distance calculates the length of the geodesic—the shortest path along the curved surface of the manifold.
As illustrated above, the Riemannian distance (blue) accounts for the curvature of the space, providing a more intrinsic measure of similarity between matrices $\Sigma_1, \Sigma_2,$ and $\Sigma_3$. This metric is uniquely affine-invariant, meaning it remains unchanged under lossless linear transformations, a property essential for identifying stable representations across different neural "read-outs."
Methods and Comparison Suite
To validate the Riemannian approach, the authors benchmarked it against a comprehensive suite of industry-standard metrics. The Riemannian distance is calculated based on the log-transformed eigenvalues of the matrices. Because distances are inherently positive, the authors implemented a condition-label permutation test to establish a "meaningful zero," allowing them to isolate true similarity from noise.
The study utilized a mix of synthetic and biological data, including simulated fMRI data to test channel-count robustness, the Haxby visual object recognition dataset for intra-subject reliability, and CIFAR-10 trained CNNs to test layer-matching accuracy across different model initializations.
Key Results
Robustness to Sampling and Nonlinearity
The researchers discovered that Riemannian distance is uniquely robust to "channel count." Whether analyzing 20 voxels or 100, the Riemannian metric provides a stable estimate of the underlying representation. Furthermore, in connectivity analysis, the metric outperformed state-of-the-art measures like Distance Correlation when relationships were nonlinear (e.g., passing through a ReLU activation).
Superior Reliability and Model Matching
In real fMRI data, Riemannian distance demonstrated significantly higher intra-subject consistency across different functional runs compared to Pearson or Spearman correlations. In the domain of artificial intelligence, when attempting to match corresponding layers across two different neural networks (trained on the same data but with different random seeds), Riemannian distance achieved significantly higher accuracy than Centered Kernel Alignment (CKA), the current gold standard.
Searchlight Sensitivity
In whole-brain "searchlight" mapping, the metric showed superior sensitivity and specificity (measured by F1-score) in localizing multivariate effects, making it a powerful tool for high-resolution brain mapping.
Conclusion and Future Directions
While the Riemannian distance offers clear advantages, it does come with constraints. Currently, the number of experimental conditions must be less than the number of response channels (voxels/neurons) to ensure matrix invertibility, though dimensionality reduction (like PCA) offers a viable workaround. Additionally, the computational cost of calculating geodesics and permutation tests is higher than simple correlations.
Future work will focus on developing faster computational estimations of null distributions and exploring the specific "curvature" of different brain regions to determine if certain areas of the manifold are more informative than others. Ultimately, these results suggest that RSA should move toward manifold-aware metrics to better summarize response patterns in both biological and artificial systems.
Summary updated