CogDPM: Diffusion Probabilistic Models via Cognitive Predictive Coding

Kaiyuan Chen, Xingzhuo Guo, Yu Zhang, Jianmin Wang, Mingsheng Long

Abstract

CogDPM bridges the gap between cognitive science and generative AI by integrating Predictive Coding (PC) theory into Diffusion Probabilistic Models (DPMs). By introducing a "Precision Weighting" mechanism that dynamically estimates data predictability, the model significantly improves spatiotemporal forecasting for complex, real-world phenomena like extreme wind and precipitation.

Introduction

Spatiotemporal forecasting—predicting how fields like weather or fluid flow evolve over time—is notoriously difficult because the "ground truth" is often chaotic and multi-scale. While deep learning has made strides, many models struggle with "blurry" predictions or fail to capture extreme events.

The authors look to the human brain for a solution. Predictive Coding (PC) is a theory suggesting our brains perceive the world by constantly matching top-down predictions against bottom-up sensory inputs. Previous attempts to build "brain-inspired" models often failed to implement a crucial component of PC: Precision Weighting. In the brain, "precision" acts as a volume knob for error signals; we pay more attention to signals that are reliable and less to those that are noisy or unpredictable. CogDPM aims to bring this dynamic "attention to uncertainty" into the world of diffusion models.

Methodology

CogDPM reinterprets the iterative denoising process of a diffusion model as a hierarchical inference process. Its architecture is built on three pillars:

  1. Hierarchical Inference: The multi-step nature of diffusion is aligned with the layers of a PC system. Early denoising steps handle holistic, "vague" structures, while later steps refine fine-grained details.
  2. Guidance as Error Correction: The model couples a "generative" DPM (representing top-down conceptual knowledge) with a "perceptual" DPM (representing bottom-up sensory stimuli). The "guidance" used to update the model's state is defined as the difference between these two—effectively a "prediction error."
  3. Precision Weighting: The model estimates its own uncertainty at every coordinate. It does this by looking at the variance of its "direct" predictions of the final state ($x_0$) over a sliding window of denoising steps. If the predictions are fluctuating wildly, the "precision" is low. The model then uses these precision maps to weight the guidance, strategically allocating more "correction intensity" to regions that are harder to predict.
CogDPM architecture bridging Predictive Coding (a) and Diffusion Models (b), with detailed step-wise guidance (c).

The paper argues that the iterative refinement of a diffusion model is a perfect mathematical analog for how the brain hierarchically amends its perception of the world. Instead of treating all errors equally, CogDPM uses the variance of its own internal states to identify "hard-to-predict" regions, such as the edges of moving objects or high-acceleration zones in a storm.

Experimental Evaluation

The authors evaluated CogDPM on a mix of synthetic and high-stakes real-world datasets, including MovingMNIST, Turbulence Flow, ERA5 Surface Wind, and UK Precipitation radar composites.

Key Results

  • Superior Accuracy: On the ERA5 wind dataset, CogDPM outperformed FourCastNet (a major industry baseline), particularly in predicting high-wind thresholds (>12 m/s and >16 m/s).
  • Better Nowcasting: In UK precipitation tasks, it surpassed DGMR (Deep Generative Model of Radar), providing more realistic intensity and better capturing the movement of squall fronts without the typical blurring.
  • Interpretable Precision: The generated "Inverse Precision" maps were found to correlate strongly with actual prediction residuals. In other words, the model successfully "knew what it didn't know," providing a built-in confidence map for forecasters.
Qualitative results on MovingMNIST, showing predictions and the corresponding inverse precision maps.

On MovingMNIST, CogDPM achieved a Continuous Ranked Probability Score (CRPS) of 0.027, matching or beating specialized RNN-based models while providing much better qualitative realism. By focusing guidance on low-precision regions, CogDPM maintains the structure and intensity of extreme events much better than previous state-of-the-art models.

Discussion and Future Work

While CogDPM shows significant promise, several avenues for future research remain. The current model is a "passive" observer; the authors note that Active Inference—where a system actively interacts with its environment to reduce uncertainty—is a major part of PC theory not yet included.

Additionally, like all diffusion models, CogDPM requires multiple inference steps, making it slower than single-pass neural networks. Real-time deployment remains a challenge, and precision estimation relies on hyperparameters (like window length and scaling factors) that may require careful tuning for different types of physical data.

Summary updated

Abstract

Predictive Coding (PC) is a theoretical framework in cognitive science suggesting that the human brain processes cognition through spatiotemporal prediction of visual world. Existing studies have developed spatiotemporal prediction neural networks based on the PC theroy, emulating its two core mechanisms: Correcting predictions from residuals and Hierarchical learning. However, these models do not show the enhancement of prediction skills on real-world forecasting tasks, and ignore the Precision Weighting mechanism of PC theory. Precision weight posits that the brain allocates more attention to signals with lower Precision, contributing to the the cognitive ability of human brains. This work introduces the Cognitive Diffusion Probabilistic Models (CogDPM) which demonstrates the connection between diffusion probabilistic models and PC theory. CogDPM features a precision estimation method based on the hierarchical sampling capabilities of diffusion models, and allocate the guidance with precision weights estimated by the inherent property of diffusion models. We experimentally show that the precision weights is an estimator of model's predictability on the rigid body and fluid motion dataset. We also apply CogDPM to real-world prediction tasks using the U.K. precipitation and ERA surface wind datasets. Our results demonstrate that CogDPM outperforms both existing domain-specific operational models and general deep prediction models in providing more proficient forecasting.
BibTeX
@inproceedings{chen2024cogdpm,
  author = {Kaiyuan Chen and Xingzhuo Guo and Yu Zhang and Jianmin Wang and Mingsheng Long},
  title = {{CogDPM: Diffusion Probabilistic Models via Cognitive Predictive Coding}},
  booktitle = {International Conference on Machine Learning},
  year = {2024},
  url = {https://openreview.net/forum?id=iLSgF7jMtI},
}
Figure