Missing data as augmentation in the Earth Observation domain: A multi-view learning approach

Francisco Alejandro Mena, Diego Arenas, Andreas Dengel

Introduction

Earth Observation (EO) increasingly relies on Multi-View Learning (MVL) to integrate diverse data streams—such as optical imagery, radar signals, meteorological data, and topographic maps—into cohesive predictive models. While multi-view approaches offer superior performance by leveraging complementary information, they are inherently vulnerable to data "missingness." In the EO domain, sensors frequently fail, cloud cover obscures optical views, and varying revisit times lead to asynchronous data availability.

Standard machine learning architectures typically require a fixed input dimension. When a view is missing, practitioners often resort to "zero-imputation," filling missing entries with zeros or mean values. However, this practice introduces artificial biases and can significantly degrade model performance. This paper presents a novel framework that treats missing data not as a failure mode, but as a form of data augmentation, coupled with dynamic feature-level fusion to ensure robustness.

Conceptual overview of the proposed Multi-View Learning (MVL) framework during training and testing phases.

Methodology

The proposed approach rests on two primary innovations: Combinations of Missing views (CoM) as Augmentation and Dynamic Feature-Level Fusion.

Combinations of Missing views (CoM) as Augmentation

Rather than training exclusively on complete data samples, the authors introduce Missing data as Augmentation (MAug). During the training phase, the model is exposed to the power set of all possible view combinations. By simulating every permutation of available and missing sensors, the model is forced to learn robust, independent representations for each modality. This strategy acts as a powerful regularizer, preventing the model from becoming overly dependent on any single high-quality view (like optical data).

Dynamic Feature-Level Fusion

To avoid the pitfalls of input-level zero-imputation, the framework employs dedicated neural network encoders for each data source. These encoders extract high-level features which are then merged using dynamic fusion functions. Unlike static concatenation, these functions can handle a variable number of input vectors, effectively "ignoring" missing views rather than processing "fake" data. The authors evaluated several fusion mechanisms:

  • Average (FCoM-av): Computes the mean of available feature vectors.
  • Gated Fusion (FCoM-ga): Uses a learnable gating mechanism to weight contributions.
  • Cross-Attention (FCoM-cr): Employs a Transformer-based attention mechanism.
  • Memory Fusion (FCoM-me): Utilizes an RNN-based sequential update.

Experimental Evaluation

The framework was tested on four diverse EO datasets:

  1. CropHarvest binary (CropH-b): Binary crop recognition.
  2. CropHarvest multi (CropH-m): 10-class crop type classification.
  3. Live Fuel Moisture Content (LFMC): Regression task for vegetation moisture.
  4. Particulate Matter 2.5 (PM25): Air quality regression.
Performance metrics (R2 scores) for LFMC and PM25 datasets as the percentage of missing views increases.

Key Results and Insights

The experimental results yielded several critical insights into the behavior of MVL models under data stress:

  • Robustness through Augmentation: Training with CoM significantly improved performance even when all data views were present. For instance, on the CropH-m dataset, the FCoM-av model achieved an F1 score of 0.686, a notable improvement over the 0.635 baseline.
  • Superiority of Dynamic Fusion: Dynamically ignoring missing views at the feature level consistently outperformed traditional imputation methods. Simple Average fusion (FCoM-av) proved remarkably effective, often matching or exceeding more complex attention-based mechanisms.
  • Regression vs. Classification: The models exhibited different failure modes across tasks. While classification performance (F1 scores) degraded gracefully under "extreme missingness," regression models (LFMC and PM25) were more sensitive, sometimes resulting in negative $R^2$ scores when only a single sensor remained available.
  • Efficiency: Because the dynamic fusion layer skips missing modalities, inference time is reduced when data is absent, unlike imputation methods that must process full-sized (albeit zeroed) tensors.

Conclusion and Future Directions

This research demonstrates that missing data in Earth Observation can be leveraged during training to build more resilient models. By combining exhaustive missing-view augmentation with dynamic feature fusion, the proposed framework achieves state-of-the-art performance across multiple EO tasks. Future work will focus on scaling the CoM approach to handle dozens of modalities and improving regression stability in extreme missingness scenarios through decision-level fusion strategies.

Summary updated

BibTeX
@article{mena2025missing,
  author = {Francisco Alejandro Mena and Diego Arenas and Andreas Dengel},
  title = {{Missing data as augmentation in the Earth Observation domain: A multi-view learning approach}},
  journal = {Neurocomputing},
  year = {2025},
  doi = {10.1016/j.neucom.2025.130175},
}
Figure