TIML: Task-Informed Meta-Learning for Agriculture

Gabriel Tseng, Hannah Kerner, David Rolnick

Introduction

Accurate machine learning models for satellite imagery are essential for global food security, enabling yield forecasting and crop mapping. However, a significant barrier exists: labeled agricultural data is heavily concentrated in high-income regions like North America and Europe. In contrast, regions like Sub-Saharan Africa, which are most vulnerable to climate change, suffer from extreme data sparsity.

While transfer learning—training on data-rich regions to fine-tune on data-sparse ones—is a common approach, standard methods often fail to leverage the rich geospatial metadata available. Agricultural characteristics are highly regional; for instance, remote sensing observations of maize in Kenya share more spectral similarities with maize in Mali than with maize in France. This paper introduces Task-Informed Meta-Learning (TIML) to bridge this gap by explicitly incorporating task-specific context into the meta-learning process.

Methodology

TIML builds upon the Model-Agnostic Meta-Learning (MAML) framework, introducing two primary innovations to handle geospatial diversity and task imbalance.

Task-Informed Meta-Learning (TIML)

Standard meta-learning typically ignores task-level metadata. TIML addresses this by employing a dedicated task encoder that processes metadata such as geographic coordinates (latitude and longitude) and the target crop type. This encoder produces a vector used to modulate the hidden layers of the main MAML model through Feature-wise Linear Modulation (FiLM).

The TIML architecture uses a task encoder to modulate model weights based on metadata like location and crop type.

A key architectural detail is that the task encoder is updated only during the outer optimization loop. This ensures that task embeddings remain stable while the model's internal weights are fine-tuned on specific local examples during the inner loop.

Forgetful Meta-Learning

Geospatial datasets often contain a mix of many "easy" tasks (e.g., binary crop vs. non-crop classification) and few "hard" tasks (e.g., identifying specific crop varieties). To prevent the model from over-optimizing for abundant easy tasks, the authors implement "Forgetful Meta-Learning." This mechanism dynamically removes training tasks from the optimization loop once the model reaches a predefined performance threshold, forcing the network to focus its capacity on more challenging, underrepresented tasks.

Experimental Results

The authors evaluated TIML on crop type classification using the CropHarvest dataset and on US county-level soybean yield estimation.

Performance in Data-Sparse Regions

TIML demonstrated superior performance across diverse geographical tasks, including Togo (crop vs. non-crop), Kenya (maize vs. rest), and Brazil (coffee vs. rest). As shown in the results below, TIML consistently outperformed standard MAML and traditional pre-training baselines.

Performance metrics across different regions show TIML's superiority in both AUC ROC and F1 scores.

Notably, in the Brazil coffee classification task—which is exceptionally difficult due to having only 26 positive examples—TIML was the only transfer learning method to significantly outperform a model trained from scratch (No pre-training). This highlights the framework's robustness in extreme few-shot scenarios where only a handful of labeled fields are available.

Yield Estimation

In regression tasks for soybean yield, TIML achieved the lowest Root Mean Square Error (RMSE) for both CNN and LSTM architectures. It even surpassed more computationally intensive baselines like Deep Gaussian Processes, suggesting that TIML is not only more accurate but also more scalable for large-scale agricultural monitoring.

Conclusion

TIML provides a powerful framework for addressing the global imbalance in agricultural data. By informing the model about where and what it is observing, and by focusing training on difficult tasks, TIML enables high-performance machine learning in regions where data is most scarce. Future applications could extend beyond agriculture to any domain featuring rich task-level metadata, such as biodiversity mapping or species distribution modeling.

Summary updated

Abstract

Labeled datasets for agriculture are extremely spatially imbalanced. When developing algorithms for data-sparse regions, a natural approach is to use transfer learning from data-rich regions. While standard transfer learning approaches typically leverage only direct inputs and outputs, geospatial imagery and agricultural data are rich in metadata that can inform transfer learning algorithms, such as the spatial coordinates of data-points or the class of task being learned. We build on previous work exploring the use of meta-learning for agricultural contexts in data-sparse regions and introduce task-informed meta-learning (TIML), an augmentation to model-agnostic meta-learning which takes advantage of task-specific metadata. We apply TIML to crop type classification and yield estimation, and find that TIML significantly improves performance compared to a range of benchmarks in both contexts, across a diversity of model architectures. While we focus on tasks from agriculture, TIML could offer benefits to any meta-learning setup with task-specific metadata, such as classification of geo-tagged images and species distribution modelling.
BibTeX
@misc{tseng2022timl,
  author = {Gabriel Tseng and Hannah Kerner and David Rolnick},
  title = {{TIML: Task-Informed Meta-Learning for Agriculture}},
  year = {2022},
  eprint = {2202.02124},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2202.02124},
}
Figure