Formalizing Open Problems to Guide Theoretical Research

34 concepts17 papers4 proposals

When a scientific discipline experiences rapid growth, it naturally produces a vast landscape of completed results. However, an exclusive focus on what has already been solved can inadvertently stall progress. Without a clear mechanism to map out the boundaries of current knowledge, a research community risks fragmentation—repeating incremental variations on well-trodden benchmarks while leaving fundamental bottlenecks unaddressed.

As Vitaly Feldman et al. observe in the Conference on Learning Theory 2016: Preface, progress in theoretical fields is frequently hindered when the community fails to explicitly identify what remains unknown. The solution is a deliberate paradigm shift: treating the identification, curation, and formalization of unsolved questions—the "known unknowns"—as a first-class scientific contribution.

The Mechanism of Open Problem Formalization

Why does formalizing an open problem accelerate a field more than simply writing a traditional literature review? The answer lies in the transition from conceptual intuition to mathematical precision. It is easy to state that a model "struggles with long-term dependencies," but it is far more powerful to define the exact mathematical constraints under which a framework fails to provide a solution.

Historically, this transition from conceptual descriptions to detailed logical proofs has been recognized as essential for the maturation of complex fields, a point argued early on by Hajnal Andréka et al. in On the Role of Mathematical Language Concept in the Theory of Intelligent Systems. In modern machine learning, many deployed applications still lack a rigorous mathematical basis. As new algorithmic challenges emerge, Philip M. Long notes that there is a persistent need to expand theoretical frameworks to cover these previously unaddressed scenarios in the Special Issue on New Theoretical Challenges in Machine Learning.

Formalization acts as a coordination mechanism. This process is what Iavor Bojinov et al. term gap-oriented synthesis in Causal Inference for Everyone. By explicitly articulating unresolved questions alongside current advancements, researchers transform passive summaries of past work into active catalysts for future discovery.

[Scattered Literature] ──> [Gap-Oriented Synthesis] ──> [Formalized Open Problem] ──> [Coordinated Community Effort]

Institutionalizing the Known Unknowns

For open problems to effectively steer a community, they must be vetted and publicized through structured channels. The theoretical machine learning community, particularly through the Conference on Learning Theory (COLT), has institutionalized this through dedicated open problem tracks.

As highlighted in the Preface to COLT 2014, the progress of theoretical research is often bottlenecked by a lack of community-vetted open problems that can direct collective effort. By establishing a formal track for these challenges, conference organizers create a strategic roadmap. According to the authors of the Preface to COLT 2024, this mechanism intentionally incentivizes the precise articulation of unsolved challenges over the presentation of completed results, fostering long-term development.

This structured curation achieves several distinct goals:

  • Identifies Intractability: It isolates specific theoretical gaps where current mathematical frameworks are fundamentally insufficient (Peter Grünwald et al. in Conference on Learning Theory 2015: Preface).
  • Establishes High-Impact Benchmarks: It prevents researchers from working in isolation on trivial problems, instead directing collective energy toward fundamental bottlenecks (Satyen Kale et al. in Preface: Conference on Learning Theory (COLT), 2017).
  • Stimulates New Frameworks: By defining rigorous benchmarks for these gaps, it forces the development of entirely new mathematical tools (Preface to COLT 2019).

This mechanism is not unique to learning theory. In computational social dynamics, Martin Hoefer et al. describe using dedicated seminar sessions to collaboratively articulate new research questions (Computational Social Dynamics (Dagstuhl Seminar 22452)). Similarly, in applied artificial intelligence, Hector Muñoz-Avila et al. introduced a "Challenge Problems" track to actively pull research out of established benchmarks and push it toward unaddressed real-world domains (IAAI-13 Preface).

Mapping the Problem Space Automatically

As the volume of scientific literature grows exponentially, manually identifying these gaps becomes increasingly difficult. Researchers face immense hurdles in validating the novelty of a proposed problem simply because they cannot comprehensively assess the existing literature base (Harshit Nigam et al. in An Interactive Co-Pilot for Accelerated Research Ideation).

Furthermore, without a structured classification of a field's problem space, it is difficult to systematically discover unexplored combinations of constraints and objective functions. As Elias Schede et al. demonstrate in A Survey of Methods for Automated Algorithm Configuration (Extended Abstract), the absence of a comprehensive map prevents researchers from recognizing structural holes that existing tools have yet to address.

To scale the formalization of open problems, recent work has turned toward automated and AI-assisted methods. For example, Priyanka Kargupta et al. developed TaxoAdapt: Aligning LLM-Based Multidimensional Taxonomy Construction to Evolving Research Corpora. By grounding Large Language Models in evolving research corpora, their framework constructs dynamic taxonomies that can proactively identify under-explored scientific domains.

Instead of merely mapping what has been written, these systems aim to automate the detection of the exact gaps between specific problems and available solutions. By transforming the discovery of open problems from a serendipitous byproduct of reading into a systematic, formalized discipline, the scientific community can ensure that its collective effort is always focused on the true frontier of knowledge.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 22 papers
  2. Version 1 · 2026-10-04 · 22 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_open_problem_formalization_e9fe6fcb,
  title = {open problem formalization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/open-problem-formalization/rdr_e9fe6fcb0cdb42dba932cb1e1310bd28},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 2, frozen text):

@misc{lacuna_open_problem_formalization_e9fe6fcb_v2,
  title = {open problem formalization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/open-problem-formalization/rdr_e9fe6fcb0cdb42dba932cb1e1310bd28/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake