Abstract
Standard Convolutional Neural Networks (CNNs) often overlook valuable non-imaging metadata, such as patient demographics or acquisition parameters, which human experts use to contextualize images. We present INSIDE (INstance modulation with SpatIal DEpendency), a novel conditioning layer that "steers" spatial attention using such metadata. By integrating a parameter-efficient Gaussian attention mechanism with feature-wise linear modulation (FiLM), INSIDE allows networks to learn localized regions of interest. Our experiments on the CLEVR-Seg and ACDC datasets show that INSIDE significantly improves segmentation performance, especially in data-scarce scenarios where metadata acts as a vital inductive bias.
Introduction
In medical imaging, scans are rarely interpreted in isolation. Clinical context—such as the laterality of a symptom or the specific slice position in a 3D volume—provides essential cues for diagnosis. While existing methods like FiLM allow metadata to scale or shift feature maps, they are spatially "blind," treating the entire image uniformly. INSIDE bridges this gap by providing conditioning layers with a spatial coordinate system, enabling the model to translate categorical metadata into precise spatial focus.
Methodology: The INSIDE Layer
The core of the INSIDE architecture is the application of a spatial attention mask to feature maps prior to standard modulation.
- Gaussian Attention: To maintain parameter efficiency and avoid overfitting, INSIDE uses a 2D Gaussian function as an attention mechanism. An auxiliary hypernetwork (a small MLP) predicts the center ($\mu$) and spread ($\sigma$) for both axes for each feature map.
- Modulation: The resulting attention mask is multiplied with the feature maps, followed by FiLM-style scaling ($\gamma$) and shifting ($\beta$).
- Training: The system is fully differentiable. An $L_2$ regularization penalty on the Gaussian's standard deviation ($\sigma$) encourages the model to learn "tight," localized attention rather than defaulting to global coverage.
Experimental Evaluation
The method was evaluated on two benchmarks:
- CLEVR-Seg: A synthetic dataset where the model segments objects based on commands like "bottom-right quadrant" or "small green cube."
- ACDC (Automated Cardiac Diagnosis Challenge): A medical dataset of cardiac cine-MRI. Metadata includes slice position and cardiac phase.
Key Results
INSIDE consistently outperforms baseline and spatially-blind conditioning methods. On CLEVR-Seg's quadrant segmentation task, INSIDE achieved a Dice score of 0.857, nearly doubling the performance of FiLM (0.487).
In the medical domain, the Gaussian attention acts as an effective "shape prior" for the heart. Notably, INSIDE demonstrates high robustness in low-data regimes. When trained on only 6% of the ACDC dataset, INSIDE maintained a Dice score of ~62%, a 16% relative improvement over the baseline (53.6%), suggesting that metadata can effectively compensate for a lack of labeled images.
Discussion and Future Work
INSIDE provides a direct "hook" for external spatial context in CNNs. While the current use of a single Gaussian per channel is efficient, it may be limited for complex, non-convex structures. Future research could explore more flexible differentiable distributions or extend the mechanism to 3D volumes by adding a third Gaussian component for the z-axis. Despite these limitations, the parameter efficiency and performance gains in low-data settings make INSIDE a promising tool for clinical applications where metadata is abundant but labeled data is scarce.
Summary updated