This blog post summarizes a discussion paper by Moritz Herrmann and Michael Herrmann regarding the work of Lutz Dümbgen and Laurie Davies (2024).
TL;DR
This paper champions a novel "model-free" approach to linear regression that calculates p-values by comparing real data against synthetic, irrelevant noise. By avoiding the assumption that a "true" mathematical model exists for any given dataset, the authors argue this method bridges a century-old cultural divide in statistics between rigid mathematical theory and pragmatic empirical reality.
What Problem Does This Paper Address?
In traditional statistics, we often start by assuming a "true" stochastic model—a mathematical formula that perfectly represents how the data was generated. The authors call this "statistical reification": the tendency to mistake our mathematical assumptions for reality itself.
This mindset has created a deep rift in the scientific community, famously described by Leo Breiman as the "two cultures." On one side are the "data modelers" who prioritize mathematical elegance and "irrevocable truth," and on the other are "algorithmic modelers" who prioritize empirical results and prediction, even if the underlying math is messy.
The problem is that when we assume a model is "The Truth," we become blind to the complexity of real-world data. This paper discusses a way to measure the relevance of variables (covariates) in a regression without needing to pretend we know the "true" model of the universe.
Methods
The paper discusses a method developed by Dümbgen and Davies that uses a unique randomization scheme:
- Gaussian Covariates: Instead of relying on complex assumptions about the data's origin, they replace observed variables with Gaussian noise—variables that are known, by definition, to be irrelevant.
- Gaussian p-values: By comparing the real data to this "known-to-be-irrelevant" noise, they derive p-values that measure a variable's importance.
- Model-Free Framework: Unlike standard permutation tests, which only become accurate as your sample size grows toward infinity (asymptotic reasoning), this approach is mathematically exact for the given data. It doesn't assume a "true" representation of data exists; it simply treats the model as a useful approximation.
Datasets
Not discussed in the paper. As a theoretical discussion and commentary, the paper focuses on the philosophical and mathematical framework rather than specific empirical benchmarks.
Key Insights and Ideas
- Bridging the "Two Cultures": The authors argue that this new approach is "orthogonal" to the traditional divide. It is formally precise (satisfying the theorists) but empirically adequate (satisfying the pragmatists).
- The Trap of Indoctrination: The authors highlight a provocative claim by Davies: that the statistical community is sometimes "indoctrinated" into rejecting unconventional methods because they don't follow established "rituals" or "norms," even when those methods are superior.
- Truth vs. Approximation: Drawing on the work of John Tukey, the authors emphasize that we should move away from "assumed truth" and toward an "approximation framework." In this view, models are tools for understanding, not mirrors of reality.
- Cultural Divides vs. Bad Intent: The authors soften the "indoctrination" critique by suggesting these issues are "cultural divides" rather than intentional malice. Conventions are often established unconsciously, making them harder to challenge.
Key Results
The primary "result" of this discussion is a conceptual validation of the Dümbgen and Davies approach:
- Formal Equivalence: The "Gaussian p-values" derived from this model-free approach are formally equivalent to traditional model-based p-values, but they are reached without the baggage of unrealistic assumptions.
- Universal Applicability: The authors suggest that this approximation framework is more robust and universally applicable than conventional statistical parlance, especially when dealing with the complexity of modern data-based inference.
Limitations and Future Work
- Lack of Recognition: The authors note that this approximation framework (originally proposed by Davies in 2014) has not yet gained the widespread attention it deserves.
- The Need for Adoption: The paper concludes that furthering this framework is essential for the evolution of statistical research, as it provides a better account of data complexity than current standards.
- Resistance to Change: A significant limitation is the "established scientific culture" itself, which may continue to resist unconventional research that falls outside of unquestioned beliefs and norms.
Summary updated