Introduction
Semantic segmentation—the process of assigning a category label to every pixel in an image—is a cornerstone of computer vision applications ranging from autonomous driving to medical diagnostics. While Vision Transformers (ViTs) have set new benchmarks in this field, they encounter a significant "resolution wall." The standard self-attention mechanism possesses quadratic complexity relative to image resolution, making high-resolution processing computationally prohibitive.
To mitigate this, architectures like the Swin Transformer employ window-based attention, restricting calculations to local sub-windows. However, this efficiency comes at the cost of "tunnel vision," where the model loses the ability to capture long-range dependencies across the image. The GLAM (GLobal Attention Multi-resolution) module addresses this by restoring global context without sacrificing the efficiency of window-based approaches.
The GLAM Module: Bridging Local and Global Context
The core innovation of GLAM is the introduction of learnable "global tokens" that act as intermediaries between isolated local windows. The process follows a structured two-step attention mechanism:
- Local Interaction (W-MSA): Within each window, visual patches interact with a set of global tokens. These tokens effectively "absorb" and summarize the local information.
- Global Interaction (G-MSA): These global tokens then interact across all windows, allowing information to propagate throughout the entire image.
Furthermore, the authors introduce Non-Local Upsampling (NLU) in the decoder stage. Unlike standard interpolation, NLU utilizes an attention mechanism to upsample features based on the global context aggregated during encoding, ensuring that fine-grained details are reconstructed with global awareness.
Experimental Evaluation
The GLAM module was evaluated across three distinct benchmarks: ADE20K (general scenes), Cityscapes (urban driving), and Synapse (medical CT scans).
Performance on Urban and General Scenes
On the Cityscapes dataset, the integration of GLAM resulted in a +1.58% improvement over the Swin-UperNet baseline. Similarly, on ADE20K, the GLAM-Swin-Unet achieved 49.10% mIoU, a 1.25-point increase over the standard architecture. These results highlight the module's ability to resolve ambiguities in complex environments by leveraging global priors.
State-of-the-Art in Medical Imaging
In the medical domain, GLAM-nnFormer set a new state-of-the-art on the Synapse dataset with an average Dice Similarity Score (DSC) of 88.60%, outperforming the previous best by 1.2 points. This suggests that global context is particularly vital for identifying anatomical structures in 3D volumetric data.
Conclusion and Future Work
The GLAM module demonstrates that global context can be efficiently reintegrated into hierarchical transformers. By using global tokens as "messengers," the architecture achieves a superior performance-per-parameter ratio compared to simply scaling up model size. Future research may explore the application of GLAM to ultra-high-resolution imagery and its optimization for 3D volumetric tasks where computational costs remain a challenge.
Summary updated