Mitigating Attention Localization in Small Scale: Self-Attention Refinement via One-step Belief Propagation

Nakyung Lee, Yeongoon Kim, Minhae Oh, Suhwan Kim, Jin Woo Koo, Hyewon Jo, Jungwoo Lee

TL;DR

This paper introduces SAOBP, a novel self-attention refinement technique that uses belief propagation to inject multi-hop dependencies, mitigating attention localization and improving model performance, especially in smaller Transformers. They also propose GTD, a metric to quantify these global interactions, offering insights into attention dynamics and model effectiveness.

What Problem Does This Paper Address?

Modern large language models (LLMs) heavily rely on the Transformer architecture, with self-attention being its core mechanism. However, a significant issue known as "attention localization" plagues these models. This means that attention distributions tend to collapse onto a small subset of tokens, failing to capture long-range dependencies effectively. This problem is particularly acute in smaller Transformer models, which have limited depth and width, hindering their ability to propagate information across layers. This localization can lead to reduced representational power, training instability, and ultimately, poorer performance on downstream tasks. The paper aims to address this by improving how self-attention captures global context, especially for resource-constrained scenarios where smaller models are preferred.

Methods

The core contribution is the Self-Attention One-step Belief Propagation (SAOBP) framework. SAOBP refines the standard self-attention mechanism by incorporating multi-hop relationships through a belief propagation (BP) process. Instead of just direct (one-hop) token interactions, SAOBP uses BP to propagate information across multiple hops. This is achieved by treating the self-attention score matrix as a factor graph and applying a single-step message-passing update.

A key novel component is the Global Token Dependency (GTD) metric. GTD quantifies the relative contribution of multi-hop connections (two or more hops) within the attention graph. This metric helps interpret and diagnose attention localization, providing a principled way to understand how well a model is leveraging global context.

The paper also introduces BP-High and BP-Low variants. BP-High uses a repulsive Potts model as a pairwise factor function to encourage diverse attention assignments, while BP-Low penalizes dissimilar label pairs to suppress diversity.

Datasets

The paper uses standard datasets for training and evaluation, including:

  • WikiText: For pretraining language models.
  • BookCorpus: For pretraining language models.
  • OpenWebText: For pretraining language models.
  • GLUE benchmark: A collection of diverse NLP tasks for fine-tuning and evaluation.
  • SQuAD: For question answering.
  • HellaSwag: For commonsense reasoning.
  • RACE-Middle: For reading comprehension.
  • SST2: For sentiment analysis.

No new benchmarks were introduced.

Key Insights and Ideas

  • Multi-hop dependencies are crucial for small models: The paper strongly suggests that explicitly modeling multi-hop token dependencies via belief propagation can significantly compensate for the architectural limitations of smaller Transformers, enabling them to achieve performance closer to larger models.
  • GTD as a diagnostic tool: GTD proves to be a valuable metric for understanding attention dynamics. It correlates well with model performance and can identify layers or heads where attention is overly localized or globally distributed.
  • Repulsive factors promote diversity: The BP-High variant, using a repulsive factor function, consistently leads to higher indirect entropy and better performance, indicating that encouraging diverse attention patterns is beneficial.
  • Trade-offs in regularization: While BP-Low also increases GTD, it can lead to skewed entropy distributions, suggesting that aggressive regularization might not always be optimal and that finding the right balance is key.
  • SAOBP is effective across architectures: The method is shown to be effective not only in encoder-style models (like BERT) but also in decoder-only architectures (like GPT-2), demonstrating its broad applicability.

Key Results

  • Performance Improvement: SAOBP, particularly the BP-High variant, consistently improves performance across various downstream tasks, especially for smaller models (BERT-Mini, BERT-Small). For instance, BP-High applied to BERT-Mini achieves accuracy comparable to or even surpassing larger models on some benchmarks.
  • Entropy Mitigation: SAOBP effectively alleviates attention entropy collapse, especially in deeper layers, leading to more stable and expressive attention distributions.
  • GTD Correlation: GTD shows significant positive correlations with model performance across multiple tasks and model sizes (|r| > 0.5, p < 0.02).
  • Attention Sparsity Reduction: SAOBP consistently reduces attention map sparsity, indicating denser and potentially more expressive attention patterns. For example, in BERT-Mini, SAOBP-High reduces sparsity from 0.906 to 0.895.
  • Computational Cost: SAOBP incurs a slight increase in per-step runtime but achieves faster loss convergence, particularly in smaller models.

Limitations and Future Work

  • Single-step BP: The current SAOBP implementation uses a single-step belief propagation. The authors suggest that exploring multi-step message passing could further enhance or potentially degrade model representational quality.
  • Fixed Repulsive Strength: The repulsive strength parameter (λ) was used with fixed values. Future work could explore layer-wise or head-wise scaling of this parameter.
  • Factor Function Exploration: While the repulsive Potts model is effective, other factor functions that enhance token interactions could be explored.
  • Adaptive Factor Functions: The authors suggest that exploring adaptive factor functions could be beneficial.

Summary updated

Abstract

Transformer-based self-attention mechanism serves as the core of modern language models, yet it often suffers from localization, where attentions collapse onto a limited subset of tokens and fail to capture long-range dependencies. To address this issue, we propose Self-Attention One-step Belief Propagation (SAOBP), a refinement framework that injects multi-hop relationships through a belief propagation process. To interpret and quantify these interactions, we introduce Global Token Dependency (GTD) that captures the relative contribution of multihop connections within the attention graph. Empirical results indicate that SAOBP helps prevent entropy collapse in deeper layers and adaptively maintains GTD at task-appropriate levels, thereby supporting improvements in model performance. Importantly, we observe competitive gains in small-scale models, highlighting its potential for improving inference quality in resource-constrained scenarios.
BibTeX
@inproceedings{lee2025mitigating,
  author = {Nakyung Lee and Yeongoon Kim and Minhae Oh and Suhwan Kim and Jin Woo Koo and Hyewon Jo and Jungwoo Lee},
  title = {{Mitigating Attention Localization in Small Scale: Self-Attention Refinement via One-step Belief Propagation}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
  url = {https://openreview.net/forum?id=4GUUkhezxk},
}
Figure