DySENSE: Dynamic Spatiotemporal ENergy Synthesis and Emulation with Satellite Imagery

Kailai Sun, Zijie Gao, Mingyi He, Can Rong, Alok Prakash, Baoshen Guo, Shenhao Wang, Jinhua ZhaoarXiv 2026 · arXiv

TL;DR

DySENSE is a multimodal generative AI framework that jointly synthesizes realistic satellite imagery and spatially aligned, monthly building energy consumption maps. By decoupling static urban form from dynamic climate-driven energy use, it enables planners to "hallucinate" energy-efficient future cities and perform "what-if" climate impact analyses with limited real-world data.

What Problem Does This Paper Address?

Cities are the front lines of the climate crisis, accounting for over 70% of global CO2 emissions. To reach net-zero targets, we need Urban Building Energy Modeling (UBEM) to design more efficient neighborhoods. However, the field faces three major hurdles:

  1. Predictive vs. Generative: Most current models can only predict energy use for existing cities. They can't easily help planners envision entirely new, energy-efficient urban layouts.
  2. Data Scarcity: High-quality, building-level energy data is notoriously hard to get due to privacy concerns and the cost of monitoring.
  3. Dynamic Complexity: Energy use isn't static; it swings wildly based on the month and local weather. Existing generative AI (GenAI) models struggle to capture these spatiotemporal "functional" dynamics.

The authors introduce DySENSE to bridge these gaps by creating a system that understands both what a city looks like from space and how its energy "pulse" changes over time.

Methods

The core innovation of DySENSE is its joint multimodal generative framework. It treats satellite imagery and energy maps as two sides of the same coin, modeled through a shared latent space.

Key components include:

  • Decoupled Branches: The model separates long-term static urban morphology (buildings, roads, parks) from short-term dynamic energy consumption (HVAC use driven by weather).
  • Dual VAEs: A standard pre-trained Variational Autoencoder (VAE) handles the RGB satellite imagery. A custom-designed Energy VAE is trained to handle the unique statistical properties of energy maps, which are often sparse and "long-tailed" (a few buildings use a lot of energy, while many use very little).
  • U-ViT Joint Denoising: A Vision Transformer-based diffusion backbone learns the joint distribution of both modalities. By sharing noise across the image and energy latents during training, the model ensures they stay perfectly aligned.
  • Multimodal Conditioning:
    • Static Prompts: Text descriptions of land-use ratios (e.g., "32% commercial, 19% residential") are embedded using CLIP.
  • Dynamic Prompts: Monthly climate variables (temperature, precipitation, solar radiation) are embedded using RemoteCLIP, a version of CLIP fine-tuned for remote sensing data.
The DySENSE architecture decouples static urban form from dynamic energy prompts to generate aligned multimodal outputs.

Datasets

The researchers introduced a new benchmark dataset called MUSE2 (Multi-city Urban Satellite-Energy).

  • Coverage: Four diverse cities—New York City, Washington D.C., Norfolk (USA), and Busan (South Korea).
  • Content: It aligns high-resolution satellite imagery from Mapbox with municipal energy disclosure records, land-use data, and gridded climate data (Daymet V4 and TerraClimate).
  • Scale: 23,880 monthly paired samples, each representing a 2km x 2km urban tile.

Key Insights and Ideas

  • Generative Data Augmentation: One of the most powerful findings is that DySENSE can act as a "data factory." By training on just 25% of real energy data and using DySENSE to generate synthetic samples, researchers achieved better results than using the full real dataset alone.
  • "What-If" Simulation: Because the model decouples form from climate, you can fix a specific generated urban layout and then "cycle" through different months or extreme weather scenarios to see how that specific design would perform.
  • RemoteCLIP Superiority: The authors found that using a text encoder specifically pre-trained on remote sensing imagery (RemoteCLIP) significantly outperformed standard CLIP for both generating images and predicting energy.

Key Results

DySENSE demonstrates impressive fidelity and utility across several metrics:

  • Downstream Boost: Using DySENSE-generated data for augmentation improved downstream energy prediction performance by 8% mIoU.
  • Error Reduction: Compared to state-of-the-art predictive models, DySENSE reduced prediction error (CVRMSE) by 9% to 18%.
  • Temporal Fidelity: The generated monthly energy trends showed a high Pearson correlation (0.96) with ground-truth data, accurately capturing the "U-shaped" seasonal energy cycles (peaks in winter/summer for heating/cooling).
  • Visual Realism: The model achieved an FID of 36.7, indicating that the generated satellite images are distributionally similar to real-world urban imagery.
Figure 2: Qualitative visualization of ground-truth and generated urban imagery and monthly energy consumption. The Energy changes are computed by the difference between monthly energy consumption values and the annual minimum values. Our generated monthly energy consumption and its trend are very similar to the ground truth.

Limitations and Future Work

While DySENSE is a major step forward, the authors acknowledge several areas for growth:

  • Geographic Scope: The current model is trained on four cities in North America and East Asia. Its performance in the Global South or different climate zones (like tropical or arid regions) remains to be tested.
  • Carbon Integration: The model focuses on energy consumption. Future iterations could integrate grid carbon intensity to directly simulate the operational carbon footprint of new urban designs.
  • Granularity: Integrating even finer-grained data, such as smart meter readings or individual building utility audits, could further refine the model's accuracy for micro-scale planning.

By releasing the MUSE2 dataset and their code as open-source, the authors hope to provide a foundation for a new era of AI-driven, energy-conscious urban design.

Summary updated

Abstract

Cities account for over two-thirds of the world's energy and more than 70% of global carbon dioxide emissions. City energy is essential for achieving the 2050 net-zero target and the UN's Sustainable Development Goals 7 and 11. Although existing studies have made progress, they are limited by data and methodological gaps. Existing studies are largely predictive for existing cities rather than generative for future cities; fine-grained, aligned monthly dynamic energy data are scarce; and existing GenAI models struggle to generate dynamic urban functional monthly energy. We present DySENSE, a joint multimodal GenAI framework that synthesizes static urban satellite imagery together with spatially aligned, dynamic monthly building energy consumption maps. We decouple long-term static urban morphology from short-term dynamic energy consumption. By conditioning on static urban land use, DySENSE can generate realistic, diverse and new static urban imagery with aligned energy maps. By conditioning on dynamic monthly and climate prompts, DySENSE can jointly generate dynamic, monthly energy consumption maps while keeping the urban morphology static. Experiments across four cities (New York, Washington, Norfolk, Busan) demonstrated that DySENSE can generate annotated high-fidelity synthetic energy data using less than 25% labelled data, boosting downstream prediction performance by 8% mIoU, and achieving strong energy dynamic consistency. Compared to existing methods, DySENSE significantly improved prediction performance (8.7% accuracy) and reduced prediction error (9%-18% CVRMSE, ASHRAE). DySENSE offers an effective foundation for energy-efficient cities, peak-load analysis, building performance simulation and dynamic demand response for city, energy, and building science. The data and code: https://huggingface.co/datasets/skl24/MUSE2 and https://github.com/kailaisun/DySENSE
BibTeX
@misc{sun2026dysense,
  author = {Kailai Sun and Zijie Gao and Mingyi He and Can Rong and Alok Prakash and Baoshen Guo and Shenhao Wang and Jinhua Zhao},
  title = {{DySENSE: Dynamic Spatiotemporal ENergy Synthesis and Emulation with Satellite Imagery}},
  year = {2026},
  eprint = {2611.17878},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.17878},
}
Figure