Normalized Space Alignment: A Versatile Metric for Representation Space Discrepancy Minimization
Lists
Only you see this
Abstract
We introduce a manifold analysis technique for quantifying the discrepancy between two representation spaces. Normalized Space Alignment (NSA) aims to compare pairwise distance between two point clouds. Our technique provides a robust means of comparing representations across different layers and models, with a particular focus on Graph Neural Networks (GNNs) to explore their unique capabilities. We show that our technique acts as a pseudometric, satisfies the properties of a similarity metric, is continuous and differentiable. We also demonstrate that NSA can serve as an effective loss function by utilizing it in autoencoders to preserve representation structure for dimensionality reduction. Furthermore, our empirical analysis showcases that NSA consistently outperforms or matches the results of previous techniques while offering computational efficiency. Its versatility extends to robustness analysis and various neural network training and representation learning applications, highlighting its wide applicability and potential to enhance the performance of neural networks.
BibTeX
@misc{ebadulla2024normalized,
author = {Danish Ebadulla and Ambuj Singh},
title = {{Normalized Space Alignment: A Versatile Metric for Representation Space Discrepancy Minimization}},
year = {2024},
url = {https://openreview.net/forum?id=5HGPR6fg2S},
howpublished = {OpenReview},
note = {Submitted to ICLR 2024},
}