Investigation into Few-Shot Instance Segmentation of Agricultural Landscapes Utilizing SAM-Driven Model

Wenbo Chen, Hongbo Zhu, Chengwei Zhao, Xuqing Li, Tingxuan Wang, Zekun Zhang, Zengyun HuInternational Journal of Energy 2025 · DOI

TL;DR

Researchers have successfully adapted Meta’s Segment Anything Model (SAM) to perform high-precision instance segmentation of agricultural fields using very little training data. By integrating SAM’s powerful feature encoder with specialized attention modules and learnable prompts, the team achieved an Average Precision (AP) of 0.875 with just 200 samples—performance that rivals traditional models trained on massive datasets.

What Problem Does This Paper Address?

In precision agriculture, we need to know exactly where one field ends and another begins to monitor crop health and manage resources effectively. This is a task for instance segmentation. However, two major hurdles stand in the way:

  1. Data Scarcity: Training deep learning models like Mask R-CNN typically requires thousands of meticulously labeled images. In specialized agricultural contexts, labeling this much data is prohibitively expensive and time-consuming.
  2. Complexity: Agricultural landscapes are messy. Fields have indistinct boundaries, varying shapes (from perfect rectangles to winding terraces), and complex backgrounds that confuse standard computer vision models.

The authors sought to determine if "foundation models"—large-scale models pre-trained on diverse data—could be "fine-tuned" to solve these problems with a fraction of the usual data (a "few-shot" approach).

Methods

The researchers explored three primary architectural enhancements based on the SAM framework:

  • Splicing SAM with Standard Heads: They replaced the standard backbones of Mask R-CNN and Mask2Former with SAM’s high-precision ViT-H (Vision Transformer) image encoder. This allowed the models to leverage SAM’s deep understanding of object boundaries from the start.
  • The SAM+ECA+Prompter Model: This is the paper's core innovation. To prevent overfitting on small datasets, they kept SAM’s parameters frozen and added:
  • Efficient Channel Attention (ECA): A lightweight module that helps the model focus on the most relevant "channels" of information (like specific textures or color gradients) without adding significant computational weight.
  • Learnable Prompter: Instead of requiring manual points or boxes to tell SAM what to segment, they trained a "Prompter" to automatically generate these cues based on the image context.
  • Training Strategy: They used the AdamW optimizer and a cosine annealing strategy to dynamically adjust learning rates, alongside L1/L2 regularization to keep the model from "memorizing" the small training set.

Datasets

The study utilized high-resolution satellite imagery from three distinct geographical regions to test the model's versatility:

  • Xichang City, China (Jilin-1 Satellite): 0.67m resolution, characterized by "blocky" farmland and clear boundaries.
  • Langfang City, China (Gaofen-2 Satellite): 1m resolution, featuring fragmented farmland and intricate details.
  • Hkamti District, Myanmar (Jilin-1 Satellite): 0.67m resolution, consisting of complex, winding terraced fields adapted to undulating terrain.

The total data volume was remarkably small: only 85, 170, and 335 samples for the three areas, respectively, split into a 6:2:2 ratio for training, validation, and testing.

Key Insights and Ideas

  • Foundation Models as Feature Extractors: The study proves that SAM’s encoder is a "universal" feature extractor for remote sensing. Even though SAM wasn't specifically trained on satellite images of corn or rice paddies, its internal representation of "what an object is" transfers beautifully to geography.
  • Freezing is Key: By freezing the main SAM parameters and only training the "Prompter" and ECA modules, the researchers avoided the common pitfall where a large model "breaks" when shown only 50 new images.
  • Geography Matters: The model performed significantly better on "blocky" fields (Xichang) than on terraces (Hkamti). This suggests that while SAM is powerful, geometric complexity still poses a fundamental challenge for few-shot learning.

Key Results

The quantitative results were impressive, showing that the model scales efficiently with very little data:

  • Extreme Few-Shot (50 samples): The models achieved an Average Precision (AP) and Average Recall (AR) of 0.715. For many agricultural applications, this is already a "usable" level of accuracy.
  • Small Batch (200 samples): Performance jumped significantly, with AP reaching 0.875 and AR surpassing 0.903.
  • Comparison: The SAM+ECA+Prompter model generally outperformed the "spliced" versions of Mask R-CNN and Mask2Former, especially in terms of boundary definition and handling small objects.

Limitations and Future Work

While the results are promising, the authors noted that terraced fields remain a "final boss" for these models. The high data complexity and varied shapes of terraces led to lower accuracy compared to flat, rectangular fields.

The authors suggest that future research should look into meta-learning (learning how to learn) and transfer learning to further bridge the gap between simple and complex landscapes. They also aim to extend this work to specific crop classification, moving beyond just finding the "field" to identifying exactly what is growing inside it.

Summary updated

Abstract

In complex agricultural landscapes, especially with limited sample sizes, current instance segmentation techniques often grapple with defining clear boundaries, thereby compromising segmentation accuracy. To mitigate this challenge, our research proposes enhancements through quantitative comparisons and the incorporation of the SAM model to bolster instance segmentation networks. Specifically, we have formulated SAM+Mask2Former and SAM+Mask R-CNN by adopting SAM's feature encoder as the foundational backbone. Further refinement is accomplished by integrating the Efficient Channel Attention (ECA) module and optimizing prompts, culminating in the SAM+ECA+Prompter model. Impressively, these models exhibit robust generalization capabilities when applied to high-resolution remote sensing imagery, even when based on smaller datasets. Empirical evidence demonstrates that, even with a mere 50 samples, the models achieve an Average Precision (AP) and Average Recall (AR) of 0.715. When scaled to 200 samples, AP surges to 0.875, while AR surpasses 0.903—figures that stand toe-to-toe with traditional field segmentation paradigms. Consequently, our research significantly elevates the efficacy of segmentation models, emphasizing a reduction in both labeling and computational expenses, thereby presenting a resource-efficient, high-precision solution for precision agriculture.
BibTeX
@article{chen2025investigation,
  author = {Wenbo Chen and Hongbo Zhu and Chengwei Zhao and Xuqing Li and Tingxuan Wang and Zekun Zhang and Zengyun Hu},
  title = {{Investigation into Few-Shot Instance Segmentation of Agricultural Landscapes Utilizing SAM-Driven Model}},
  journal = {International Journal of Energy},
  year = {2025},
  doi = {10.54097/9nqvrz65},
}
Figure