Position: Not All Explanations for Deep Learning Phenomena Are Equally Valuable

Alan Jeffares, Mihaela van der SchaarICML 2025 · OpenReview

TL;DR

This paper argues that many "deep learning phenomena" (like double descent or grokking) are often studied in isolation with narrow, ad hoc explanations that don't translate to real-world applications. Instead, researchers should leverage these phenomena as unique testbeds to refine broader, more generalizable theories of deep learning.

What Problem Does This Paper Address?

The deep learning research community has become increasingly fascinated with "phenomena" – surprising observations that seem to defy our current understanding of how neural networks work. Examples include double descent (where test error decreases again after increasing model complexity), grokking (delayed generalization after perfect training), and the lottery ticket hypothesis (finding small, efficient subnetworks). While these phenomena are popular and generate many papers, the authors argue that the common approach of developing isolated, case-by-case explanations for them is often inefficient. These explanations rarely reflect real-world challenges and may not contribute meaningfully to the broader field. The paper aims to shift the focus from solving isolated puzzles to using these phenomena as opportunities to build more robust, generalizable theories of deep learning.

Methods

This paper is a position piece, not an empirical study. Its "method" is primarily argumentative and analytical. The authors:

  • Analyze the current research landscape: They observe the prevalence of research on deep learning phenomena and the common approach of developing ad hoc hypotheses.
  • Critique the value of isolated explanations: They argue that explanations that are "overfit" to a specific phenomenon lack broader utility and predictive power.
  • Propose an alternative framework: They advocate for a "sociotechnical pragmatism" approach, where research value is rooted in downstream impact. This means using phenomena to refine "broad explanatory theories" that encapsulate fundamental deep learning principles.
  • Illustrate with examples: They discuss double descent, grokking, and the lottery ticket hypothesis, analyzing their practical relevance and potential for broader theoretical contributions.
  • Offer practical recommendations: They suggest ways researchers and the community can adopt a more intentional and pragmatic approach to studying these phenomena.

Datasets

Not discussed in the paper. The paper focuses on theoretical arguments and analysis of existing research, rather than presenting new empirical results on specific datasets.

Key Insights and Ideas

  • "Phenomena" are often edge cases: Many studied phenomena don't appear prominently in practical, large-scale deep learning applications.
  • Ad hoc hypotheses are often low-utility: Explanations tailored to a single phenomenon, without broader implications, are akin to solving a puzzle rather than advancing fundamental understanding.
  • Value lies in refining broad theories: Deep learning phenomena are best utilized as unique settings to test, refine, and falsify general theories (e.g., bias-variance tradeoff, optimization principles).
  • Pragmatism is key: Researchers should consider the potential downstream utility of their work, asking how an explanation might inform broader principles or practical applications, rather than just solving the immediate puzzle.
  • Scientific rigor is essential: Adopting a more scientific approach, including preregistration and reconciliation of findings with existing literature, can lead to more robust and valuable insights.

Key Results

As this is a position paper, there are no quantitative experimental results. The "results" are conceptual:

  • A critique of the current research norms surrounding deep learning phenomena.
  • An argument for prioritizing research that refines broad explanatory theories over narrow, ad hoc explanations.
  • Practical recommendations for researchers to adopt a more pragmatic and scientifically rigorous approach to studying these phenomena.

Limitations and Future Work

  • Subjectivity in assessing utility: The authors acknowledge that determining an explanation's exact location on the spectrum between ad hoc and broad is subjective.
  • Predicting future utility is hard: While they advocate for pragmatic considerations, they admit that predicting which explanatory theories will be most useful a priori is challenging.
  • Broader applicability: The paper suggests that the principles discussed can extend beyond "edge case" phenomena to other areas of deep learning, such as emergent abilities in large language models.
  • Call for community dialogue: The authors hope their work sparks a broader conversation about how to best pursue research on deep learning phenomena and maximize their value.
  • Practical guidelines: The paper outlines practical considerations and a self-evaluation checklist for researchers to align their work with the proposed pragmatic approach.

Summary updated

Abstract

Developing a better understanding of surprising or counterintuitive phenomena has constituted a significant portion of deep learning research in recent years. These include double descent, grokking, and the lottery ticket hypothesis -- among many others. Works in this area often develop *ad hoc hypotheses* attempting to explain these observed phenomena on an isolated, case-by-case basis. This position paper asserts that, in many prominent cases, there is little evidence to suggest that these phenomena appear in real-world applications and these efforts may be inefficient in driving progress in the broader field. Consequently, we argue against viewing them as isolated puzzles that require bespoke resolutions or explanations. However, despite this, we suggest that deep learning phenomena *do* still offer research value by providing unique settings in which we can refine our *broad explanatory theories* of more general deep learning principles. This position is reinforced by analyzing the research outcomes of several prominent examples of these phenomena from the recent literature. We revisit the current norms in the research community in approaching these problems and propose practical recommendations for future research, aiming to ensure that progress on deep learning phenomena is well aligned with the ultimate pragmatic goal of progress in the broader field of deep learning.
BibTeX
@inproceedings{jeffares2025position,
  author = {Alan Jeffares and Mihaela van der Schaar},
  title = {{Position: Not All Explanations for Deep Learning Phenomena Are Equally Valuable}},
  booktitle = {International Conference on Machine Learning},
  year = {2025},
  url = {https://openreview.net/forum?id=cw7MYyDL33},
}

Loading related content...

Figure