Abstract
Benefiting from the capability of building interdependencies among channels or spatial locations, attention mechanisms have been extensively studied and broadly used in a variety of computer vision tasks recently. In this paper, we investigate light-weight but effective attention mechanisms and present triplet attention, a novel method for computing attention weights by capturing cross-dimension interaction using a three-branch structure. For an input tensor, triplet attention builds inter-dimensional dependencies by the rotation operation followed by residual transformations and encodes inter-channel and spatial information with negligible computational overhead. Our method is simple as well as efficient and can be easily plugged into classic backbone networks as an add-on module. We demonstrate the effectiveness of our method on various challenging tasks including image classification on ImageNet-1k and object detection on MSCOCO and PASCAL VOC datasets. Furthermore, we provide extensive insight into the performance of triplet attention by visually inspecting the GradCAM and GradCAM++ results. The empirical evaluation of our method supports our intuition on the importance of capturing dependencies across dimensions when computing attention weights. Code for this paper can be publicly accessed at https://github.com/LandskapeAI/triplet-attention.
Introduction
In modern computer vision, attention mechanisms serve to guide neural networks toward salient features. Existing approaches like Squeeze-and-Excitation (SE) networks focus on channel-wise attention ("what"), while others like the Convolutional Block Attention Module (CBAM) combine channel and spatial attention ("where"). However, these methods often treat channels and spatial dimensions independently, collapsing one to compute weights for the other. This creates a "dimensionality gap," where the rich interaction between specific channels and spatial coordinates is lost unless expensive 3D convolutions are employed.
Triplet Attention addresses this by capturing "cross-dimension interactions" through a lightweight, rotation-based mechanism. By rethinking the tensor dimensions, it learns complex dependencies with nearly zero parameter overhead.
Methodology: The Triplet Attention Module
The core innovation of Triplet Attention is its three-branch structure that utilizes rotation to capture dependencies across all three dimensions of a tensor: Channel ($C$), Height ($H$), and Width ($W$).
Cross-Dimension Interaction via Rotation
Instead of processing only the standard $(C, H, W)$ tensor, the module creates three parallel paths. As illustrated in Figure 1, these branches capture interactions between different pairs of dimensions.
- Branch 1 (Spatial): Standard spatial attention capturing dependencies between $H$ and $W$.
- Branch 2 (Channel-Height): Rotates the tensor to $(W, H, C)$ to capture interactions between the Channel and Height dimensions.
- Branch 3 (Channel-Width): Rotates the tensor to $(H, C, W)$ to capture interactions between the Channel and Width dimensions.
Architectural Comparison and Efficiency
To maintain efficiency, each branch employs a "Z-pool" layer, which concatenates max-pooled and average-pooled features across the first dimension. This reduces the dimension to two channels while preserving significant feature information. A $7 \times 7$ convolution followed by a Sigmoid activation then generates the attention mask.
As shown in Figure 5, Triplet Attention (d) differs from predecessors like SE (a) and CBAM (b) by avoiding Multi-Layer Perceptrons (MLPs) and dimensionality reduction ratios. This design choice prevents the loss of non-linear local dependencies that typically occurs when compressing channel information.
Experimental Results
The effectiveness of Triplet Attention was validated across multiple benchmarks:
- ImageNet-1k Classification: Adding Triplet Attention to a ResNet-50 backbone improved Top-1 accuracy by 2.04%. Remarkably, it adds only 4,800 parameters, whereas SE and CBAM add approximately 2.5 million parameters for comparable gains.
- Object Detection (MSCOCO & PASCAL VOC): In detection tasks using Faster R-CNN, Triplet Attention consistently outperformed baseline models and other attention modules in mean Average Precision (mAP) while maintaining a significantly smaller parameter footprint.
- Visual Inspection: Grad-CAM heatmaps demonstrated that models equipped with Triplet Attention focus more precisely on target objects, producing tighter and more accurate activations compared to CBAM or baseline architectures.
Conclusion
Triplet Attention introduces a simple yet powerful method for capturing cross-dimension dependencies in CNNs. By leveraging rotation and Z-pooling, it bridges the gap between channel and spatial attention without the computational burden of 3D convolutions or large MLPs. Its parameter-efficient nature and ease of integration make it an ideal "plug-and-play" module for a wide range of computer vision tasks.
Summary updated