Certify, Report, or Refuse: Harm-Constrained Conformal Prediction for Clinical Actions

Zhiyuan Li, Lequan YuarXiv 2026 · arXiv

TL;DR

HC-ROCP is a new framework for clinical decision support that goes beyond simple accuracy to directly constrain harmful medical errors. By using conformal prediction with a specific "harm penalty," it can rigorously certify when a model's clinical policy is safe, provide descriptive fallbacks when evidence is thin, or outright refuse to make claims for rare subgroups.

What Problem Does This Paper Address?

In medicine, not all errors are equal. Sending a patient with a benign mole home is fine; sending a patient with melanoma home for "routine follow-up" is a catastrophe. Standard AI metrics like "top-1 accuracy" or "average loss" often fail to capture these high-stakes risks, especially for rare conditions.

Furthermore, even if a model looks safe on a small test set, how do we know that safety will hold up in the real world? There is a critical gap between "low observed error" and "statistically guaranteed safety." This paper addresses the need for "harm-constrained" predictions that come with rigorous statistical certificates, particularly when dealing with rare, severe diseases where data is scarce.

Methods

The authors introduce Harm-Constrained Risk-Optimal Conformal Prediction (HC-ROCP). This is a "training-free" framework, meaning it can be applied to any existing, "frozen" medical classifier.

The core innovation is adding a harm penalty to the decision-making process. When the model chooses a set of possible diagnoses and a corresponding clinical action (like "biopsy" or "urgent referral"), it explicitly penalizes actions that would be harmful if a "severe" condition were present.

To ensure the safety claims are statistically sound, the framework uses a three-stage pipeline:

  1. Construct Split: Proposes a family of candidate policies by varying coverage and harm parameters.
  2. Coverage Split: Tests if the predicted sets actually contain the true diagnosis at the required rate (e.g., 95%).
  3. Safety Split: Audits the frequency of severe-action errors.
Overview of the HC-ROCP framework

Crucially, it uses multiplicity adjustment (specifically Clopper-Pearson bounds with Bonferroni correction) to account for testing many different policies and subgroups simultaneously. This allows the system to issue one of several evidence labels, ranging from "High-confidence certified" to "Insufficient evidence" for specific subgroups.

Evidence labels

Datasets

The framework was evaluated on three public medical imaging benchmarks:

  • COVID-19 chest radiographs: Used to test the trade-off between action error and modeled loss.
  • ISIC 2019 dermoscopy: Used to test evidence for rare subgroups (e.g., Melanoma vs. Squamous Cell Carcinoma).
  • EyePACS-to-APTOS: A diabetic retinopathy transfer split used to test how well the system recalibrates after a "domain shift" between different clinical settings.

Key Insights and Ideas

  • The "Data Gap" for Certification: A striking finding is the "zero-error requirement." To certify a 1% harm budget with high confidence, you might need over 1,200 severe cases with zero observed errors. This provides a concrete target for how much more data clinicians actually need to collect to trust an AI's safety claims.
  • Global vs. Subgroup Safety: A model can be "safe on average" while still being dangerous for rare subgroups. HC-ROCP is honest about this, potentially certifying a general population policy while "refusing" to claim safety for rare conditions like Squamous Cell Carcinoma (SCC) due to small sample sizes.
  • Action-Aware Uncertainty: Unlike standard conformal prediction, which just provides a list of possible labels, HC-ROCP connects those labels directly to clinical actions and their associated risks.

Key Results

  • Dramatic Error Reduction: On COVID-19 data, HC-ROCP slashed severe-action errors from 4.68% down to 0.54% compared to standard baselines.
  • Certification Thresholds: At a 5% harm budget, global policies for COVID-19 and ISIC were successfully certified. However, at a stricter 1% budget, no dataset had enough severe cases to meet the statistical bar for certification, resulting in "expected-risk" descriptive reports instead.
Selected operating points
  • The Cost of Safety: Reducing harm isn't free; it typically leads to higher "modeled loss," which in a clinical sense means more referrals or more intensive testing for uncertain cases.
COVID-19 frontier (loss vs. harm)

Limitations and Future Work

The authors acknowledge that this is a retrospective study on static data. It doesn't account for real-world factors like clinician overrides, hospital capacity constraints, or the actual impact on patient outcomes. The "loss matrices" used to define harm were provided by experts but haven't been validated in a live clinical trial.

Future work aims to compare different underlying classifiers within this same framework and to develop better ways to screen candidate policies to reduce the "multiplicity penalty" that comes from testing too many options. The authors also suggest that these certification tests could be a starting point for prospective "silent-mode" clinical studies.

Summary updated

Abstract

Routing a melanoma case to routine follow-up is a serious action error that diagnostic coverage alone does not rule out. We present Harm-Constrained Risk-Optimal Conformal Prediction (HC-ROCP), a training-free framework that uses frozen classifier probabilities and adds a harm penalty when choosing prediction sets and actions. We test the resulting policies on disjoint calibration splits with multiplicity adjustment. HC-ROCP certifies supported claims, reports descriptive fallback policies, and refuses unsupported subgroup claims. We evaluate these outputs retrospectively on COVID-19 radiographs, ISIC 2019 dermoscopy, and EyePACS-to-APTOS retinopathy transfer. Across ten calibration seeds, HC-ROCP reduces COVID-19 severe-action errors from the exact risk-optimal baseline's 4.68\% to 0.54\%. At a 5\% budget, global claims are certified for COVID-19 and ISIC, but the rare squamous-cell-carcinoma claim is refused. At 1\%, all settings receive descriptive reports. For each budget, we also report the minimum severe-case count needed to pass the global harm gate with zero errors, providing a starting point for further data collection.
BibTeX
@misc{li2026certify,
  author = {Zhiyuan Li and Lequan Yu},
  title = {{Certify, Report, or Refuse: Harm-Constrained Conformal Prediction for Clinical Actions}},
  year = {2026},
  eprint = {2611.12662},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.12662},
}
Figure