MissScore: High-Order Score Estimation in the Presence of Missing Data

Wenqin Liu, Haoze Hou, Erdun Gao, Biwei Huang, Qiuhong Ke, Howard Bondell, Mingming GongSubmission to ICLR 2025 · Rejected

Abstract

The first order derivative (score) of data density, typically estimated via denoising score matching, has emerged as an effective tool for modeling data distribution and generating synthetic data. Extending this concept to higher-order scores could uncover more detailed local information of the data distribution, enabling new applications. However, learning these high-order scores usually requires complete data, which is often unavailable in real-world scenarios such as healthcare and finance due to privacy and cost constraints. In this work, we introduce MissScore, a novel score-based framework for learning high-order scores from observations with missing data. We derive objective functions for estimating high-order scores under different missing data mechanisms and propose a new algorithm to handle missing data effectively. Our empirical results demonstrate that MissScore efficiently and accurately approximates high-order scores with missing data, while enhancing sampling speed and data quality, as validated through several downstream tasks, including data generation and causal discovery.
BibTeX
@inproceedings{liu2025missscore,
  author = {Wenqin Liu and Haoze Hou and Erdun Gao and Biwei Huang and Qiuhong Ke and Howard Bondell and Mingming Gong},
  title = {{MissScore: High-Order Score Estimation in the Presence of Missing Data}},
  booktitle = {International Conference on Machine Learning},
  year = {2025},
  url = {https://openreview.net/forum?id=NHTqYrp5YL},
}
Figure