TL;DR
This paper introduces a method to transform opaque contextual word embeddings (like those from BERT) into a more interpretable format, where each dimension explicitly represents a semantic category. By achieving high accuracy in supersense prediction tasks with these transformed embeddings, the authors demonstrate a way to make complex language models more transparent and understandable.
What Problem Does This Paper Address?
Modern NLP models, particularly those based on transformers like BERT and GPT-3, have achieved remarkable success. However, their internal representations, or "embeddings," are notoriously difficult to interpret. These embeddings are high-dimensional vectors where individual dimensions don't have clear, human-understandable meanings. This "black box" nature makes it challenging to understand why a model makes certain predictions and hinders debugging and trust. The paper aims to bridge this gap by creating a transformation that maps these dense, opaque embeddings into a space where dimensions correspond to interpretable semantic properties, specifically using WordNet's supersense categories.
Methods
The core of the paper is a novel transformation process designed to make contextual embeddings interpretable. Here's a breakdown:
- Sparse Representation (Optional): The authors first explore using sparse coding to obtain sparse representations from dense contextual embeddings. This step is optional and aims to mimic how humans often describe concepts using a limited set of properties.
- Whitening: To decorrelate semantic features and standardize the embedding space, a ZCA whitening transformation is applied. This step is crucial for preparing the data for the subsequent semantic analysis.
- Semantic Distribution Analysis: The key idea is that words belonging to the same semantic category should have similar distributions of their embedding coefficients along certain dimensions. The authors define two distributions for each dimension i and semantic category j: $P_{ij}$ (coefficients of words in category j) and $Q_{ij}$ (coefficients of words not in category j).
- Hellinger Distance for Transformation Matrix: A transformation matrix $W$ is constructed where each entry $W(i, j)$ quantifies the distance between $P_{ij}$ and $Q_{ij}$. The Hellinger distance is chosen for its continuous, symmetric, and bounded nature. This distance measures how well dimension i encodes semantic category j.
- Bias Reduction and Directional Encoding: To address potential biases from imbalanced categories and ensure semantic categories are represented in their corresponding basis, the transformation matrix is normalized and then modified to encode directionality (positive or negative influence).
- Post-processing with Frequencies: Finally, the transformed representations are adjusted by incorporating the frequency of supersense categories in the training data, ensuring that the likelihood of each category is reflected.
Datasets
- SemCor: Used for training the transformation matrix and learning sparse coding dictionaries.
- SensEval2, SensEval3, SemEval 2007 Task 17, SemEval 2013 Task 12, SemEval 2015 Task 13: These are standard sense-annotated corpora, concatenated into an "ALL-dev" dataset for evaluation. The focus is on fine-grained sense annotations at the "lexname" (supersense) level.
- SemEval 2007 data: Used as a development set for hyperparameter tuning.
Key Insights and Ideas
- Interpretable Dimensions: The fundamental insight is that by analyzing the distribution of embedding coefficients across semantic categories, one can construct a transformation that maps original dimensions to explicit semantic features. This moves beyond opaque latent spaces.
- Hellinger Distance Effectiveness: The choice of Hellinger distance proved effective for quantifying the semantic information encoded in each dimension, offering a robust measure compared to alternatives like Bhattacharyya distance.
- Sparse Representations Amplify Semantics: Using sparse representations, either as input or through the transformation, consistently amplified the semantic information, leading to better performance on the supersense prediction task. This suggests that sparsity aligns well with human-like semantic decomposition.
- SenseBERT's Advantage: Models like SenseBERT, which are pre-trained with explicit supersense information, benefit significantly from this transformation, demonstrating that prior semantic knowledge can be further amplified.
- Multilingual Transfer: The approach shows promise for multilingual settings, as demonstrated by successful supersense prediction on languages other than English using a transformation learned on English data.
Key Results
- High Accuracy in Supersense Prediction: The proposed interpretable representations achieved competitive accuracy on the supersense prediction task, often matching or exceeding other interpretable methods like SMS (Sparsity Makes Sense). For instance, using BERT Base with the proposed approach achieved accuracies in the mid-60s to low-70s, and even higher for SenseBERT (around 74-79%).
- Sparse Representations Improve Performance: Across various models, relying on sparse representations (either as input or through the transformation) generally led to higher accuracy compared to dense representations.
- Dimensionality Reduction is Effective: Even when reducing the dimensionality of embeddings significantly (from 768/1024 to 45), the performance did not drop drastically, indicating that the transformation effectively captures salient semantic information. The proposed method outperformed PCA-based dimension reduction.
- Case Sensitivity Matters: For some models like BERT and DistilBERT, using cased vocabularies yielded better results for supersense prediction.
- Qualitative Analysis: Visualizations (Figure 2) showed that the transformed representations effectively cluster words by semantic categories, with correct predictions being visually distinguishable.
Limitations and Future Work
- Limited Dimensionality: The authors acknowledge that by reducing dimensions to 45, they are inherently limiting the model's capacity compared to the original dense embeddings. They do not expect to exceed the original model's performance but aim for interpretability.
- Hyperparameter Sensitivity: The performance of sparse representations is dependent on hyperparameters like regularization coefficient ($\lambda$) and number of basis vectors ($k$), which require tuning.
- Future Work: The paper suggests that the method can be extended to other NLP tasks beyond supersense prediction, such as term expansion and knowledge base completion. Further quantitative experiments are also needed to solidify findings in multilingual domains. The classification decision is currently based on the maximum coefficient, and exploring alternative decision mechanisms could be a future direction.
Summary updated