Understanding the null hypothesis (H0) in non-inferiority trials

Jihad MallatCritical 2017 · DOI · OpenAlex

Author: Jihad Mallat
Source: Critical Care, BioMed Central

Abstract

Non-inferiority trials (NITs) are increasingly common in clinical research, yet their statistical interpretation remains a source of confusion. This paper provides a methodological critique of how "non-inferiority" is reported, specifically addressing the common error of claiming treatment superiority from a trial designed only to show non-inferiority. By examining the fundamental differences in null hypotheses and the critical role of confidence intervals, this article clarifies the standards required for robust clinical conclusions.

Introduction

In clinical research, the gold standard is often the superiority trial, designed to prove that a new intervention is more effective than the current standard. However, in many cases, a new treatment may offer advantages such as lower cost, fewer side effects, or easier administration, even if its efficacy is not strictly superior. In these instances, researchers employ non-inferiority trials (NITs) to demonstrate that the new treatment is "not unacceptably worse" than the standard of care.

A significant challenge arises when researchers attempt to pivot from a non-inferiority finding to a claim of superiority without the appropriate statistical framework. This critique, authored by Jihad Mallat, highlights these pitfalls using a recent study on septic shock resuscitation as a case study.

The Statistical Framework: Superiority vs. Non-Inferiority

The core difference between these trial designs lies in their starting assumptions, or null hypotheses ($H_0$).

  1. Superiority Trials:
    • $H_0$: There is no difference between the two treatments.
  2. Goal: Reject $H_0$ to demonstrate that a statistically significant difference exists.
  3. Non-Inferiority Trials (NIT):
    • $H_0$: The new treatment is worse than the standard treatment by at least a pre-defined margin, known as the non-inferiority margin ($\Delta$).
  4. Goal: Reject $H_0$ to prove that the treatment is not worse than the standard by more than $\Delta$.

Because the null hypotheses are fundamentally different, the interpretation of p-values and errors (Type I and Type II) cannot be interchanged. A significant p-value in an NIT ($p < 0.05$) only indicates that the treatment has met the criteria for being "not worse"; it does not inherently mean the treatment is "better."

Interpreting Outcomes via Confidence Intervals

To accurately determine the clinical significance of an NIT, researchers must look beyond the p-value to the 95% confidence interval (CI) of the treatment difference.

Figure 1: Scenarios for interpreting results in a non-inferiority trial based on 95% confidence intervals.

As shown in Figure 1, there are three primary scenarios for interpreting the 95% CI in relation to the non-inferiority margin ($-\Delta$) and the line of no difference (0):

  • Inconclusive: The CI crosses the non-inferiority margin ($-\Delta$). We cannot rule out that the new treatment is unacceptably worse.
  • Non-inferiority shown but Superiority not shown: The entire CI is to the right of $-\Delta$, but it still includes zero. This confirms the treatment is not worse, but it is not significantly better.
  • Superiority shown: The entire CI is to the right of both $-\Delta$ and zero. Only in this case can superiority be claimed.

Case Analysis: The "Lactate Strategy" in Septic Shock

Mallat applies this framework to a study by Zhou et al. (2017), which compared a "lactate-guided" resuscitation strategy to a standard "ScvO2-guided" strategy. The original authors reported a lower mortality rate in the lactate group (18.3% vs. 27.9%, $p = 0.033$) and concluded that the lactate strategy was superior.

However, Mallat points out several methodological flaws:

  1. Misinterpreted P-value: The $p = 0.033$ was a p-value for non-inferiority. It confirms the lactate strategy is not worse, but it does not statistically support the claim of superiority.
  2. Missing Confidence Intervals: The original study failed to report the 95% CI for the difference in mortality. Without this interval, it is impossible to verify if the results exclude zero and thus demonstrate superiority.
  3. Arbitrary Margins: The researchers chose a non-inferiority margin ($\Delta$) of 15% without clinical justification, whereas previous landmark trials in the field (e.g., Jones et al., 2010) utilized a more conservative 10% margin.

Conclusion and Recommendations

The distinction between non-inferiority and superiority is not merely semantic; it is a fundamental statistical requirement. To ensure the integrity of critical care research, Mallat recommends the following:

  • Justify the Margin: The non-inferiority margin ($\Delta$) must be rigorously justified based on clinical relevance and previous literature.
  • Report Confidence Intervals: Authors must always provide 95% CIs to allow readers to assess both non-inferiority and potential superiority.
  • Maintain Design Integrity: Trials should be analyzed according to their original design. Claims of superiority arising from NITs must meet the strict criterion of the 95% CI excluding zero.

By adhering to these standards, researchers can avoid the "p-value trap" and provide clearer, more reliable evidence for clinical practice.

Summary updated

Abstract

Fig. 1 Different possible scenarios of the results of a non-inferiority clinical trial. is the non-inferiority margin
BibTeX
@inproceedings{mallat2017understanding,
  author = {Jihad Mallat},
  title = {{Understanding the null hypothesis (H0) in non-inferiority trials}},
  booktitle = {Critical},
  year = {2017},
  doi = {10.1186/s13054-017-1685-2},
}
Figure