Abstract
Hyperspectral imaging (HSI) captures hundreds of narrow, contiguous wavelength bands, making it a powerful tool in biology, agriculture, and environmental monitoring. However, interpreting Vision Transformers (ViTs) in this setting remains largely unexplored due to two key challenges: (1) existing saliency methods struggle to capture meaningful spectral cues, often collapsing attention onto the class token, and (2) full-spectrum ViTs are computationally prohibitive for interpretability, given the high-dimensional nature of HSI data. We present FOCUS, the first framework that enables reliable and efficient spatial-spectral interpretability for frozen ViTs. FOCUS introduces two core components: class-specific spectral prompts that guide attention toward semantically meaningful wavelength groups, and a learnable [SINK] token trained with an attraction loss to absorb noisy or redundant attention. Together, these designs make it possible to generate stable and interpretable 3D saliency maps and spectral importance curves in a single forward pass, without any gradient backpropagation or backbone modification. FOCUS improves band-level IoU by 15 percent, reduces attention collapse by over 40 percent, and produces saliency results that align closely with expert annotations. With less than 1 percent parameter overhead, our method makes high-resolution ViT interpretability practical for real-world hyperspectral applications, bridging a long-standing gap between black-box modeling and trustworthy HSI decision-making.
Introduction
Hyperspectral imaging (HSI) provides a "spectral fingerprint" for objects, which is critical for high-precision tasks like plant disease detection. While Vision Transformers (ViTs) excel at processing this high-dimensional data, they often function as "black boxes." Traditional saliency methods like Grad-CAM, designed for 3-channel RGB images, suffer from "spectral blindness" and "attention collapse" when applied to HSI. These methods often produce blurry spatial maps that ignore the spectral dimension or collapse attention onto non-informative tokens. Furthermore, the computational cost of generating explanations for hundreds of spectral bands is often prohibitive.
FOCUS (Fused Observation of Channels for Unveiling Spectra) addresses these challenges as a lightweight, gradient-free framework that enables high-resolution spatial-spectral interpretability for frozen ViT backbones.
Methodology
FOCUS introduces three novel components to a frozen ViT backbone (e.g., DINOv2) to generate precise 3D saliency cubes in a single forward pass:
- Class-Specific Spectral Prompts: Instead of generic tokens, FOCUS utilizes learnable prompts tailored to specific classes and spectral groups (e.g., Visible, Red-edge, NIR, SWIR). These act as spectral anchors, guiding the model toward biologically meaningful wavelength groups.
- The [SINK] Token & Attraction Loss: To mitigate attention collapse, a dedicated
[SINK]token is introduced. A lightweight "attraction loss" encourages non-informative or redundant attention to flow into this sink, effectively filtering noise from the final saliency map. - 3D Prompt-CAM Generation: By aggregating attention from spectral prompts to spatial patches, FOCUS generates a unified $H \times W \times C$ saliency cube without requiring multiple gradient backpropagation passes.
Experimental Results
The framework was evaluated on the HyperLeaf 2024 and Tomato BLS datasets. FOCUS demonstrated significant improvements over traditional methods, including a 15% increase in band-level Intersection-over-Union (IoU) and a 40% reduction in attention collapse.
Qualitatively, FOCUS provides much clearer localization of features across different spectral bands. As shown in the visual results, while Grad-CAM and Prompt-CAM produce diffuse heatmaps, FOCUS accurately identifies specific spectral peaks—such as the 750nm red-edge crucial for plant stress detection—and isolates them from noise in the SWIR bands.
Beyond accuracy, FOCUS is highly efficient. It achieves a 95% reduction in runtime and an 85% reduction in memory usage compared to standard ViT interpretability methods, all while adding less than 1% parameter overhead (0.8M vs. 86M for a ViT-B/16).
Conclusion
FOCUS bridges the gap between high-performance ViT modeling and trustworthy decision-making in hyperspectral imaging. By "domesticating" the attention sink phenomenon and grounding interpretability in domain-specific spectral groups, it provides a practical, efficient solution for real-world agricultural and biological monitoring. Future work will focus on scaling these prompts for large-scale classification tasks and refining noise-suppression mechanisms.
Summary updated