Decision Theory via Model-Free Generalized Fiducial Inference

Jonathan P. Williams, Yang LiuLecture Notes in Computer Science 2024 · DOI · OpenAlex

Abstract: Building on the recent development of the model-free generalized fiducial (MFGF) paradigm (Williams, 2023) for predictive inference with finite-sample frequentist validity guarantees, in this paper, we develop an MFGF-based approach to decision theory. Beyond the utility of the new tools we contribute to the field of decision theory, our work establishes a formal connection between decision theories from the perspectives of fiducial inference, conformal prediction, and imprecise probability theory. In our paper, we establish pointwise and uniform consistency of an MFGF upper risk function as an approximation to the true risk function via the derivation of nonasymptotic concentration bounds, and our work serves as the foundation for future investigations of the properties of the MFGF upper risk from the perspective of new decision-theoretic, finite-sample validity criterion, as in Martin (2021).

Introduction

Traditional decision theory is built upon the minimization of risk, defined as the expected loss under an assumed probability model. However, the validity of such decisions is inherently tied to the accuracy of the underlying model. In high-stakes scenarios, model misspecification can lead to suboptimal or even dangerous outcomes. While empirical risk minimization offers a distribution-free alternative, it often lacks robust uncertainty quantification. This paper introduces a novel framework for decision theory based on the Model-Free Generalized Fiducial (MFGF) paradigm, bridging the gap between distribution-free inference and imprecise probability.

The MFGF Framework

The proposed approach leverages the MFGF paradigm to provide predictive inference with finite-sample frequentist guarantees. Central to this method is the use of nonconformity measures—borrowed from conformal prediction—to assess the "unusualness" of data points. Instead of a single probability distribution, the MFGF approach generates a "credal set," which is a collection of probability measures compatible with the observed data.

From this credal set, the authors derive the MFGF upper risk function (or upper prevision). This function represents the maximum possible risk across all distributions in the credal set, providing a "worst-case" but statistically valid estimate of risk. In certain simple cases, this method simplifies to the classic non-parametric Dempster-Hill procedure.

Illustration of the GF transducer and loss function used to define the MFGF upper risk.

Theoretical Guarantees

A key contribution of this work is the establishment of pointwise and uniform consistency for the MFGF upper risk. Through the derivation of nonasymptotic concentration bounds, the authors prove that the upper risk converges to the true risk at a rate of approximately $1/n$. This ensures that while the estimate is naturally conservative (overestimating risk to ensure safety), it is not asymptotically wasteful; as the sample size grows, the upper risk estimate tightens around the true risk.

Numerical Illustration

To validate the theory, the authors conducted a synthetic simulation using data generated from a standard normal distribution truncated to the interval $[-3, 3]$. They employed a standard squared error loss function, $\ell(\vartheta, y) = (y - \vartheta)^2$, and evaluated the performance for sample sizes $n=20$ and $n=200$.

Results and Discussion

The simulation results demonstrate the practical utility of the MFGF upper risk. The upper prevision successfully bounds the true risk function, with the bound becoming increasingly precise as the sample size increases.

Convergence of the MFGF upper risk to the true risk (left) and the distribution of risk minimizers (right) for sample sizes n=20 and n=200.

The right panel of the figure above illustrates the distribution of the "upper prevision minimizers"—the decisions made by minimizing the MFGF upper risk. These decisions are centered on the true optimal value (zero), and their variance decreases significantly as the sample size grows, confirming the method's reliability and accuracy.

Conclusion

This work establishes a formal "triple threat" connection between fiducial inference, conformal prediction, and imprecise probability. By providing a model-free framework with finite-sample guarantees, it offers a robust foundation for future decision-theoretic research. Future investigations will focus on extending these results to non-convex loss functions and unbounded data domains, as well as exploring specific finite-sample validity criteria.

Summary updated

BibTeX
@inproceedings{williams2024decision,
  author = {Jonathan P. Williams and Yang Liu},
  title = {{Decision Theory via Model-Free Generalized Fiducial Inference}},
  booktitle = {Lecture Notes in Computer Science},
  year = {2024},
  doi = {10.1007/978-3-031-67977-3\_14},
}

Loading related content...

Figure