Rethinking the Construction of Effective Metrics for Understanding the Mechanisms of Pretrained Language Models

You Li, Jinhui Yin, Yuming LinEMNLP 2023 · DOI

Introduction: The Probing Dilemma

To decode the "black box" of models like BERT, researchers typically employ probes—diagnostic models designed to extract linguistic features from internal vector representations. However, traditional probing faces a significant trade-off:

  1. Supervised Probes: Rely on human-annotated labels (e.g., Penn Treebank), risking the possibility that the probe itself learns the task rather than reflecting the model's inherent knowledge.
  2. Unsupervised Metrics: Use geometric properties like cosine similarity, which are interpretable but often too simplistic to capture complex hierarchical structures.

This paper introduces the tree topological probe, a self-supervised framework that bridges this gap by establishing mathematical bounds on traditional structural metrics, allowing for a more rigorous understanding of how models encode syntax.

Methodology: The Tree Topological Probe

Building on the structural probe by Hewitt and Manning (2019), the authors map embeddings to a space where the $L_2$ norm represents word depth in a syntax tree. To move beyond simple error measurement, they introduce two self-supervised bounds:

  • $X_{ssp}$ (Lower Bound): The distance to the "best possible" tree-like sequence ($mins_W$).
  • $X_{essp}$ (Upper Bound): The distance to the "worst possible" tree-like sequence ($maxs_W$).
Geometric representation of the tree topological probe bounds $X_{ssp}$ and $X_{essp}$.

By analyzing where supervised metrics fall within these bounds, the researchers can distinguish between generic tree encoding and the memorization of specific linguistic patterns. They also propose an unbiased optimal geometry metric to account for varying tree depth distributions without assuming a specific linguistic preference.

Key Insights and Experimental Results

The study, conducted primarily on BERT-large and RoBERTa-large using the $rtd_{mix}$ benchmark, yielded several critical insights into the internal mechanisms of PLMs.

The "U-Shaped" Syntactic Curve

Contrary to the intuition that models get progressively "smarter" in deeper layers, BERT exhibits a peak in syntactic encoding in its middle layers (around layer 8). As shown in the metric evolution plot, these layers are most effective at representing generic, unbiased tree structures.

Evolution of syntactic metrics across BERT layers, illustrating the peak in syntactic encoding in middle layers and the bounding behavior of the proposed metrics.

Memorization vs. Generalization

In later layers (14–18), the model shifts from general hierarchical rules to "memorizing" specific real-world data distributions. Here, self-supervised and supervised metrics converge, indicating a loss of general syntactic flexibility. By the final layers, "syntactic saturation" occurs, where the cost of memorizing specific tree structures actually leads to a decrease in the accuracy of the linear encoding for general syntax.

Performance Gains via Regularization

The authors leveraged these insights to improve fine-tuning. By using $X_{ssp}$ as a regularization loss on Layer 10 (the "rising phase" of syntax learning) during the CoLA task, they increased the Matthew's Correlation Coefficient (MCC) from 63.34 to 64.87. Conversely, applying this to later "saturated" layers (12–19) caused model failure (MCC of 0.00), highlighting the importance of layer-specific intervention.

Conclusion

The tree topological probe provides a rigorous "sanity check" for interpretability by mathematically bounding traditional metrics. While the current construction of optimal sequences relies on greedy approximations, future work may utilize Category Theory for a more formal mathematical foundation. Additionally, the sensitivity of these metrics to root depth definitions remains an area for further refinement.

Summary updated

BibTeX
@inproceedings{li2023rethinking,
  author = {You Li and Jinhui Yin and Yuming Lin},
  title = {{Rethinking the Construction of Effective Metrics for Understanding the Mechanisms of Pretrained Language Models}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2023},
  doi = {10.18653/v1/2023.findings-emnlp.894},
}
Figure