Abstract
This study evaluates the conservatism of the standard Student’s $t$-interval $\bar{X} \pm t_{\alpha/2} s/\sqrt{n}$ for the unknown mean $\theta$ when the underlying distribution $F$ is non-normal. By analyzing the infimum of coverage probabilities across symmetric unimodal distributions, we identify a critical threshold for robustness. While high-confidence intervals (e.g., 95%, 99%) remain remarkably stable or conservative, intervals with low nominal coverage (where $t_{\alpha/2} < 1$) exhibit a collapse in reliability, with minimum coverage probabilities dropping to zero.
Introduction
A fundamental assumption in classical statistics is that data are sampled from a normal distribution. Under this assumption, the $t$-interval provides an exact confidence interval for the mean. While practitioners often rely on the Central Limit Theorem to justify the use of $t$-intervals for non-normal data, this asymptotic justification may fail for small to moderate sample sizes. This paper investigates the "worst-case" performance of the $t$-interval when the underlying distribution is symmetric and unimodal but potentially non-Gaussian.
Methodology
The authors determine the infimum coverage probability of the $t$-interval within the class of all symmetric unimodal distributions. The analysis utilizes the Khintchine representation, which allows any unimodal distribution to be expressed as a mixture of uniform distributions. This simplifies the optimization problem into finding the "worst" mixture. Additionally, independent embedding techniques and numerical minimization are employed to calculate exact coverage bounds for sample sizes ranging from $n=2$ to $n=10$.
The Robustness Threshold and the $t=1$ Discontinuity
A primary discovery is the existence of a sharp performance "cliff" related to the critical value $t$. For intervals where the critical value $t$ is significantly greater than 1, the $t$-interval is robust and often conservative. However, as the nominal confidence level decreases, there is a point $t_n$ below which the infimum coverage probability becomes zero. This means that for certain non-normal but symmetric and unimodal distributions, a low-confidence interval may have zero probability of containing the true mean.
As shown in the table above, for a sample size of $n=10$, the interval maintains its integrity as long as the critical value $t$ is above 2.25.
Quantitative Performance
For standard scientific confidence levels, the $t$-interval performs admirably. In the family of symmetric unimodal distributions, the minimum actual coverage for a nominal 95% interval is very close to the target, even for small samples. This suggests that the $t$-test is "conservative" in high-stakes scenarios—it is safer than advertised.
The data in Table 1 indicates that for $n=10$, a 95% nominal interval has a minimum actual coverage of 94.5%. Even at $n=2$, the 95% interval provides at least 92% coverage. This confirms that for high-confidence applications, the $t$-test is a robust choice. However, the table also illustrates how the infimum begins to diverge from the nominal value as the confidence level (and thus $t$) decreases.
Conclusion
The Student’s $t$-interval is surprisingly robust to departures from normality, provided the target confidence level is high. For 95% and 99% intervals, the actual coverage is often higher than or very close to the nominal value. The primary risk occurs at lower confidence levels or when interpreting $P$-values in large samples with slight contamination, where the test can become overly sensitive to tiny departures from normality. Researchers should remain cautious when using $t$-statistics for low-confidence estimations or in the presence of significant outliers.
Summary updated