Simplicity First Search for Automated Hypothesis Selection

28 concepts19 papers1 proposal

When an artificial intelligence system attempts to explain the world—whether it is diagnosing a medical patient, discovering a chemical reaction pathway, or parsing the narrative arc of a story—it faces an immediate mathematical crisis: the combinatorial explosion of possible explanations. For any given set of observations, there is an infinite number of hypotheses that can logically entail them. A system that searches blindly through this infinite space will quickly stall out.

To make automated theory formation computationally tractable, researchers have long relied on a mechanized version of Occam’s Razor. Rather than treating simplicity merely as a tie-breaker between two equally good theories, the core insight of this research direction is to use simplicity as an active, structural filter to guide the search process itself. By organizing the hypothesis space from the simplest configuration to the most complex, systems can find valid explanations efficiently while avoiding overfitting.

The most direct way to implement a simplicity bias is to structure the search space as a ladder of increasing complexity. Instead of starting with a fully fleshed-out, highly detailed model and pruning away unnecessary parts, the system begins with the absolute minimum representation required to formulate a hypothesis. It only introduces more complex relations, parameters, or entities when the simpler models are explicitly refuted by evidence.

[ Minimal Model ] ---> Fits Data? ---> YES ---> [ Terminate & Return ]
       |
       NO (Refuted by Data)
       v
[ Increment Complexity by One Step ] ---> Fits Data? ---> ...

This "bottom-up" approach drastically reduces the number of states the search algorithm must visit. As Buchanan et al. demonstrated in their pioneering work on scientific theory induction, A Heuristic Programming Study of Theory Formation in Science, heuristic pruning based on simplicity constraints is what prevents a system from drowning in experimental noise.

This incremental strategy manifests across several distinct domains:

  • Chemical Pathways: In the MECHEM system, Valdés-Pérez showed in Theory-Driven Discovery of Reaction Pathways in the MECHEM System that an infinite search space of chemical reactions becomes manageable if the algorithm incrementally increases the number of steps and chemical species only when simpler hypotheses fail to explain the experimental data.
  • Physical Simulations: Iwasaki et al. applied this to physical systems in Automated Model Selection for Simulation, proving that starting with the most abstract representation of a phenomenon and adding detail on demand is far more computationally efficient than starting with a maximal model.
  • Pattern Recognition: Early formalizations by Zagoruyko et al. in Hypothesis of Simplicity in Pattern Recognition and the authors of Hypothesis established that evaluating algebraic systems in strict order of increasing structural complexity resolves classification ambiguities at lower levels before they can overcomplicate the model.
  • Neural Network Topology: Even in modern connectionist architectures, Gomez et al. demonstrated in Compressed Network Complexity Search that biasing the initial search distribution toward simpler network topologies and higher weight compression ratios consistently yields parameter-efficient models that generalize better.

This same principle underpins broader scientific discovery frameworks, such as the GELL-MANN system detailed in Incremental Discovery of Hidden Structure: Applications in Theory of Elementary Particles, which automates the postulation of hidden components (like quarks) by strictly prioritizing the simplest structural explanations that fit observable particle data.

Logical Factoring and Unification

Simplicity is not just about counting parameters; it is also about unification. A theory is parsimonious if it explains multiple seemingly independent observations through a single underlying cause.

In abductive logic—the process of generating the best explanation for a set of facts—this is achieved through a process called logical factoring. As Pople explained in On the Mechanization of Abductive Logic, an explanatory system can apply Occam's Razor by actively merging different branches of a search tree. When the system detects that two separate symptoms can be traced back to a single common root cause, it collapses those search paths into one. This synthesis keeps the explanatory framework as lean as possible.

We see this exact mechanism mirrored in natural language processing. When a machine reads a story, it must infer the unstated goals of the characters. Granger demonstrated in When Expectation Fails: Towards a Self-Correcting Inference System that an intelligent system maintains stability by favoring interpretations where the fewest number of inferred goals account for the largest number of observed actions.

Minimizing Entities and Redundancy

A critical constraint within parsimonious search is the prevention of "phantom" entities. When a logical system encounters a gap in its knowledge, a trivial way to bridge that gap is to invent a new, unobserved entity to fulfill the requirement. However, this quickly leads to bloated, unfalsifiable theories.

To counter this, automated induction systems use explicit pruning strategies to penalize the creation of unnecessary entities. In first-order logic induction, Morgan introduced a mechanized method in Automated Hypothesis Generation Using Extended Inductive Resolution that heavily penalizes existential quantifiers and Skolem functions. This forces the system to ground its hypotheses in existing data rather than inventing unnamed entities to explain away anomalies.

Similarly, Demolombe et al. formalized this in An Inference Rule for Hypothesis Generation by focusing the search entirely on "minimal supports"—the most general, non-redundant assumptions sufficient to prove a goal.

This entity-minimization strategy is also vital for spatial and common-sense reasoning. As Adorni et al. showed in Some Basic Mechanisms for Common Sense Reasoning About Stories Environments, when a system constructs a mental model of a physical space from a narrative, it must exhaustively attempt to satisfy spatial relationships using existing objects before it is permitted to instantiate a new one. This ensures that subsequent descriptions of an environment resolve to the same conceptual instances rather than creating redundant duplicates.

Conversely, if an existing model is already too complex, parsimony can be driven in reverse via data-driven compression. Adjei et al. demonstrated in Refining Learning Maps with Data Fitting Techniques: Searching for Better Fitting Learning Maps that a greedy iterative search can systematically simplify complex hierarchical graphs by merging adjacent nodes when empirical data fails to distinguish between them, resulting in a more compact and predictive model.

Why Simplicity Drives Performance

Why does this persistent bias toward parsimony work so well across entirely different domains of AI?

First, it acts as a powerful filter against noise. As Brown noted in Steps Toward Automatic Theory Formation, enforcing a preference for minimal disjunctions allows automated systems to isolate universal structural rules while ignoring the idiosyncratic, dataset-specific anomalies that lead to overfitting.

Second, parsimonious search aligns computational efficiency with human utility. In expert systems and logic programming, an explanation is only useful if a human operator can understand it. Walker demonstrated in Prolog/Ex 1, An Inference Engine Which Explains Both Yes and No Answers that optimizing the internal heuristic search to find the shortest possible proof tree directly yields highly readable, concise explanations. By eliminating redundant logical steps during the search itself, the system ensures that its final output is both mathematically sound and human-interpretable.

Key Foundational Work

A Heuristic Programming Study of Theory Formation in Science (1971) — contributed 2 concept elements to this direction.
Hypothesis of Simplicity in Pattern Recognition (1971) — contributed 2 concept elements to this direction.
On the Mechanization of Abductive Logic (1973) — contributed 2 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_parsimonious_hypothesis_selection_84e669d8,
  title = {parsimonious hypothesis selection},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/parsimonious-hypothesis-selection/rdr_84e669d8adbd4bc78a7e8a1a6e42857d},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 1, frozen text):

@misc{lacuna_parsimonious_hypothesis_selection_84e669d8_v1,
  title = {parsimonious hypothesis selection},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/parsimonious-hypothesis-selection/rdr_84e669d8adbd4bc78a7e8a1a6e42857d/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake