SSLA: A Generalized Attribution Method for Interpreting Self-Supervised Learning without Downstream Task Dependency
Lists
Only you see this
Abstract
Self-Supervised Learning (SSL) is a crucial component of unsupervised tasks, enabling the learning of general feature representations without the need for labeled categories. However, our understanding of SSL tasks remains limited, and it is still unclear how SSL models extract key features from raw data. Existing interpretability methods are heavily reliant on downstream tasks, requiring information from these tasks to explain SSL models. This reliance blurs the line between interpreting the SSL model itself and the downstream task model. Moreover, these methods often require additional samples beyond the target of interpretation, introducing extra information that complicates the interpretability process. In this paper, we propose three fundamental prerequisites for the interpretability of SSL tasks and design the Self-Supervised Learning Attribution (SSLA) algorithm that adheres to these prerequisites. SSLA redefines the interpretability objective by introducing a feature similarity measure, reducing the impact of randomness inherent in SSL algorithms, and achieving more stable interpretability results. Additionally, SSLA abstracts the interpretability process, making it independent of specific neural network architectures. To the best of our knowledge, SSLA is the first SSL interpretability method that does not rely on downstream tasks. We also redesign a more reasonable evaluation framework and establish baselines for comparative assessment. The source code for our implementation is publicly available at https://anonymous.4open.science/r/SSLA-EF85.
BibTeX
@misc{zhu2025ssla,
author = {Zhiyu Zhu and Jiayu Zhang and NAN YANG and Xinyi Zhang and Zhibo Jin and Jianlong Zhou and Fang Chen},
title = {{SSLA: A Generalized Attribution Method for Interpreting Self-Supervised Learning without Downstream Task Dependency}},
year = {2025},
url = {https://openreview.net/forum?id=2bEjhK2vYp},
howpublished = {OpenReview},
note = {Submitted to ICLR 2025},
}Loading related content...