No article text was recorded for this version.
attention collapse (attention head redundancy)
Version 2 of 2114 papers125 conceptsfrozen 2026-10-04
Papers in this version
- Blog: Transformers in computational visual media: A survey
- Blog: Devil is in the Uniformity: Exploring Diverse Learners within Transformer for Image Restoration
- Blog: Channel Propagation Networks for Refreshable Vision Transformer
- Blog: Multi-Head Decoder for End-to-End Speech Recognition
- Blog: A Theoretical Analysis of Self-Supervised Learning for Vision Transformers
- Blog: Gramformer: Learning Crowd Counting via Graph-Modulated Transformer
- Blog: Infinite Limits of Multi-head Transformer Dynamics
- Blog: Diversified Visual Attention Networks for Fine-Grained Object Classification
- Blog: Learning where to look: Semantic-Guided Multi-Attention Localization for Zero-Shot Learning
- Blog: Devil is in the Uniformity: Exploring Diverse Learners within Transformer for Image Restoration
- Blog: Efficient Multi-Model Fusion with Adversarial Complementary Representation Learning
- Blog: Multi-focus Attention Network for Efficient Deep Reinforcement Learning
- Blog: Dual Optimal Transport for Multi-Concept Composition: Structural Alignment and Texture Injection in Diffusion Models
- Blog: Semantic Matching and Aggregation Network for Few-shot Intent Detection
- Blog: Revisiting Structured Dropout
- Blog: Progressive Self-Supervised Attention Learning for Aspect-Level Sentiment Analysis
- Blog: Shatter and Gather: Learning Referring Image Segmentation with Text Supervision
- Blog: Multi-Head Attention with Diversity for Learning Grounded Multilingual Multimodal Representations
- Blog: Diversity-Based Generalization for Neural Unsupervised Text Classification under Domain Shift
- Blog: Diversified Visual Attention Networks for Fine-Grained Object Classification
- Blog: Comparator Networks
- Blog: State Representation Learning Using an Unbalanced Atlas
- Blog: Diverse Part Discovery: Occluded Person Re-Identification With Part-Aware Transformer
- Blog: Diversity Regularized Spatiotemporal Attention for Video-Based Person Re-Identification
- Blog: Repulsive Attention: Rethinking Multi-head Attention as Bayesian Inference
- Blog: CQA-Face: Contrastive Quality-Aware Attentions for Face Recognition
- Blog: MHSA-Net: Multihead Self-Attention Network for Occluded Person Re-Identification
- Blog: Coarse2Fine: A Two-stage Training Method for Fine-grained Visual Classification
- Blog: Multi-Head Multi-Loss Model Calibration
- Blog: Rethinking Token Reduction with Parameter-Efficient Fine-Tuning in ViT for Pixel-Level Tasks
- Blog: Dynamic Prototype Mask for Occluded Person Re-Identification
- Blog: On Biasing Transformer Attention Towards Monotonicity
- Blog: A Deep Convolutional Neural Network for Pneumonia Detection in X-ray Images with Attention Ensemble
- Blog: Get The Point of My Utterance! Learning Towards Effective Responses with Multi-Head Attention Mechanism
- Blog: MHSAN: Multi-Head Self-Attention Network for Visual Semantic Embedding
- Blog: Lightweight Attentional Feature Fusion: A New Baseline for Text-to-Video Retrieval
- Blog: Hierarchical Pyramid Diverse Attention Networks for Face Recognition
- Blog: Regularized Attentive Capsule Network for Overlapped Relation Extraction
- Blog: Dynamic Graph Representation Learning for Video Dialog via Multi-Modal Shuffled Transformers
- Blog: Cluster-to-Conquer: A Framework for End-to-End Multi-Instance Learning for Whole Slide Image Classification
- Blog: A Multi-sentiment-resource Enhanced Attention Network for Sentiment Classification
- Blog: ABD-Net: Attentive but Diverse Person Re-Identification
- Blog: Slot-VLM: Object-Event Slots for Video-Language Modeling
- Blog: A Primal-Dual Framework for Transformers and Neural Networks
- Blog: Explainable Depression Detection using Masked Hard Instance Mining
- Blog: Multi-Level Structured Self-Attentions for Distantly Supervised Relation Extraction
- Blog: Hybrid Distillation: Connecting Masked Autoencoders with Contrastive Learners
- Blog: Robust Facial Landmark Detection by Cross-order Cross-semantic Deep Network
- Blog: Adaptive Template Transformer for Mitochondria Segmentation in Electron Microscopy Images
- Blog: Revisiting Audio-Visual Segmentation with Vision-Centric Transformer
- Blog: Transformers Provably Learn Directed Acyclic Graphs via Kernel-Guided Mutual Information
- Blog: Low Rank Factorization for Compact Multi-Head Self-Attention
- Blog: Infinite Limits of Multi-head Transformer Dynamics
- Blog: Learning Semantic-Aligned Feature Representation for Text-based Person Search
- Blog: Deep Convolutional Pooling Transformer for Deepfake Detection
- Blog: Enhancing Sentence Embedding with Generalized Pooling
- Blog: The Principle of Diversity: Training Stronger Vision Transformers Calls for Reducing All Levels of Redundancy
- Blog: Part-Aware Attention Network for Person Re-identification
- Blog: Multi-Attentional Deepfake Detection
- Blog: Revealing the Dark Secrets of Masked Image Modeling
Showing 60 of 114 papers.
History & citation
Current · Permanent link
Cite this direction
Latest version (a living page):
@misc{lacuna_attention_collapse_attention_head_redund_53b73421,
title = {attention collapse (attention head redundancy)},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/attention-collapse-attention-head-redundancy/rdr_53b73421c63942559a72bb4161ffafc6},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}This exact version (version 2, frozen text):
@misc{lacuna_attention_collapse_attention_head_redund_53b73421_v2,
title = {attention collapse (attention head redundancy)},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/attention-collapse-attention-head-redundancy/rdr_53b73421c63942559a72bb4161ffafc6/v/2},
note = {Version 2, frozen 2026-10-04. Accessed 2026-11-22},
}