Abstract
In agricultural remote sensing, the integration of domain-specific knowledge—such as Vegetation Indices (VIs)—into deep learning architectures remains a significant challenge. This paper introduces the Generalized Vegetation Index (GVI), a learnable module designed to optimize spectral band combinations, and Additive Group Normalization (AGN), a novel normalization layer that stabilizes the fusion of multi-modal data. By applying these methods to the Agriculture-Vision dataset, we demonstrate a 2% mIoU improvement over standard baselines, particularly in identifying vegetation-related agricultural patterns. Our results suggest that providing specific inductive biases tailored to remote sensing data can significantly enhance the performance of semantic segmentation models in precision agriculture.
Introduction
Agricultural monitoring via satellite and aerial imagery utilizes more than just the standard Red-Green-Blue (RGB) spectrum. Near-Infrared (NIR) and thermal bands are critical for assessing crop health and soil conditions. For decades, domain experts have utilized manually engineered Vegetation Indices (VIs), such as the Normalized Difference Vegetation Index (NDVI), to quantify these properties. However, integrating these non-linear, domain-specific indices into modern convolutional neural networks (CNNs) presents two primary hurdles: the difficulty of manual parameter tuning for diverse datasets and the potential disruption of ImageNet-pretrained weights when introducing new, highly non-linear input channels.
Methodology
Generalized Vegetation Index (GVI)
To address the limitations of hard-coded indices, we propose the GVI module. We observed that most common VIs share a mathematical structure based on the ratio of linear combinations of spectral bands. The GVI formalizes this as a learnable, model-agnostic operation:
$$GVI(\mathbf{x}, \alpha, \beta) = \frac{\mathbf{x} \circledast \alpha}{\mathbf{x} \circledast \beta}$$
where $\circledast$ denotes a convolution operation. This allows the network to learn the optimal index for a specific segmentation task, effectively treating domain knowledge as a powerful inductive bias.
Additive Group Normalization (AGN)
Standard Batch Normalization (BN) often fails to capture features that require normalization across channels, a common requirement for VIs. While Group Normalization (GN) is better suited for this, replacing BN entirely can degrade the performance of models relying on ImageNet-pretrained weights. AGN bridges this gap by combining both:
$$\hat{\mathbf{x}}^{(AGN)} = \sigma(\rho) \cdot \hat{\mathbf{x}}^{(GN)} + \hat{\mathbf{x}}^{(BN)}$$
By initializing the learnable parameter $\rho$ to a large negative value, the model initially relies on the stable, pretrained BN and gradually incorporates GN as it learns to process the new GVI features.
Experimental Setup and Results
Dataset
We evaluated our approach using the Agriculture-Vision dataset, which consists of 21,061 aerial images (512x512 resolution) across four channels (RGB + NIR). The dataset includes complex labels such as Cloud Shadow, Double Plant, Planter Skip, Standing Water, Waterway, and Weed Cluster.
Performance Analysis
Using a DeepLabV3 architecture with an EfficientNet backbone, our proposed GVI + AGN framework achieved an mIoU of 46.87%, representing a nearly 2% absolute improvement over the 44.92% baseline.
Significant gains were observed in vegetation-specific classes. For instance, the IoU for "Standing Water" increased from 51.03% to 57.14%, and "Weed Cluster" improved from 49.48% to 51.28%. Qualitatively, the model demonstrated a robust ability to identify subtle patterns like "Double Plant" lanes, even in instances where ground-truth annotations were incomplete.
Conclusion
This work demonstrates that bridging the gap between traditional remote sensing domain knowledge and modern deep learning can yield substantial performance benefits. The GVI module provides a structured way for models to learn relevant spectral relationships, while AGN ensures that these new modalities do not interfere with the benefits of transfer learning. Future research will focus on the scalability of these modules across diverse global climates and their compatibility with a wider range of neural backbones.
Summary updated