Introduction
OlmoEarth introduces a new family of multimodal foundation models specifically engineered for Earth Observation (EO). By implementing a streamlined training regime known as Latent MIM Lite, researchers have achieved state-of-the-art performance across nearly 30 diverse tasks. Crucially, this approach resolves the stability issues—such as representation collapse—that frequently hinder latent-space modeling in complex visual domains.
The Challenge of Earth Observation Data
Modeling Earth Observation data presents unique difficulties compared to standard computer vision. EO data is inherently spatial (image-like), sequential (time-series of satellite passes), and highly multimodal, often requiring the integration of optical light, radar, and elevation maps.
Existing foundation models for EO face two primary hurdles:
- Training Instability: Advanced techniques that model images in abstract "latent space" often suffer from representation collapse, where the model fails to learn meaningful distinctions between different geographic features.
- The Accessibility Gap: Many cutting-edge models are computationally prohibitive for the non-profits and NGOs that require them for critical missions like conservation, food security, and disaster response.
Methodology: Latent MIM Lite
The core innovation of OlmoEarth is Latent MIM Lite (Latent Masked Image Modeling of Linear, Invariant Token Embeddings). While standard Masked Image Modeling (MIM) attempts to reconstruct missing pixels, more advanced versions try to reconstruct abstract features using a dynamic "target encoder." OlmoEarth simplifies this by using a frozen, randomly initialized linear projection to generate target features. This "Lite" approach provides the representative power of latent modeling without the instability of a moving target encoder.
Other key technical choices include:
- Modality-Aware Masking: To prevent the model from exploiting redundancies between different sensors (e.g., radar and optical data of the same area), OlmoEarth masks entire modalities or time-steps, forcing the model to learn deeper cross-modal relationships.
- Bandset-Specific Loss: By restricting contrastive loss to specific subdivisions of satellite data ("bandsets"), the model avoids "easy negatives" and learns more robust, discriminative features.
- Scalable Architecture: The models utilize a Vision Transformer (ViT) encoder-decoder architecture, ranging from "Nano" (1.4M parameters) to "Large" (300M parameters).
Global-Scale Pre-training
The model was pre-trained on a global dataset of 285,288 locations, each containing a full one-year time series. The input modalities included Sentinel-1 (radar), Sentinel-2 (multispectral), and Landsat-8. To ground the model in physical reality, it was also tasked with reconstructing derived maps such as WorldCereal (crop types), SRTM (elevation), and Canopy Height Maps.
Evaluation was conducted against 12 other foundation models using 18 research benchmarks (including GEO-Bench) and 19 real-world datasets provided by partners such as the African Wildlife Foundation and NASA JPL.
Performance and Real-World Impact
OlmoEarth establishes a new "Pareto optimum" for EO models, delivering significantly better performance for the same compute budget compared to predecessors like Clay or CROMA.
Key results include:
- Feature Extraction: The model achieved top performance on 15 out of 24 tasks when used as a frozen feature extractor (kNN/Linear Probing).
- Fine-tuning: When fully fine-tuned, OlmoEarth led the field in 19 out of 29 tasks.
- Case Studies: The model demonstrated immediate utility in the field, achieving a 98.1% F1 score for mangrove mapping and state-of-the-art accuracy for ecosystem classification in North Africa.
Beyond the model itself, the OlmoEarth Platform provides an end-to-end tool for non-experts to label data, fine-tune models, and run inference, directly addressing the adoption gap in the humanitarian sector.
Future Directions
While OlmoEarth represents a significant step forward, several areas for growth remain. The "Large" model did not always outperform the "Base" version, suggesting that training recipes for high-parameter EO models require further refinement. Future iterations aim to incorporate weather and climate data (such as ERA5) and higher-resolution aerial imagery to provide even more granular environmental context.
Summary updated