Solving Ill-Defined Problems Through Co-Evolution

17 concepts9 papers1 proposal

In classical artificial intelligence and computer science, problem-solving is often framed as a search through a maze. You start at a clearly defined initial state, apply a sequence of valid operators, and navigate toward a known goal state. This paradigm works beautifully for games like chess or well-structured optimization problems where the rules and constraints are set in stone.

But consider real-world engineering, architectural design, or software development. When a client asks a designer to "remodel a bathroom" or an organization asks an engineer to "optimize a machine learning pipeline," there is no clean maze. The initial state is messy, the operators are open-ended, and the success criteria are vague and subjective.

For decades, researchers have recognized that these ill-defined problems cannot be solved by traditional search algorithms alone. Instead, solving them requires a dynamic, iterative process where the problem specification and the solution co-evolve.


The Myth of the Static Constraint Space

Traditional computational models often treat problem-solving as a static constraint satisfaction problem. The assumption is that if you can just gather all the requirements upfront, a system can compute the optimal solution.

However, as Freeman et al. pointed out, this approach breaks down in complex design domains. In these environments, the design space is fundamentally undefined at the outset. You cannot gather all the constraints before you begin because the constraints themselves only emerge as the solution progresses.

When a problem lacks a formal representation language or clear success criteria, traditional information processing models struggle, as noted by Eastman. If an automated system expects a perfectly structured mathematical input, it will fail when a human user provides incomplete, program-like hints or vague objectives, a challenge highlighted by Gorgely et al..


The Specification Process: How Situations "Talk Back"

If the problem space isn't fixed, how do humans solve these tasks? The key differentiator lies in an active, cognitive specification process.

According to an early foundational study on architectural design, the solver does not merely search an existing space; they actively construct the problem space and its goals through memory retrieval and iterative refinement (Cognitive (1969)). This creates a conversation between the solver and the artifact.

┌──────────────────────────┐       Proposes Partial Solution       ┌──────────────────────────┐
│  Problem Specification   │ ────────────────────────────────────> │    Solution Artifact     │
│ (Vague goals, contexts)  │ <──────────────────────────────────── │  (Sketches, prototypes)  │
└──────────────────────────┘       Reveals New Constraints         └──────────────────────────┘
                                   ("The Situation Talks Back")

This interplay is known as co-evolution. As explored by Lemke et al., a designer might start with a loose set of requirements and propose a partial solution—such as a rough user interface sketch or a preliminary software architecture. Building this partial solution forces the designer to confront concrete realities.

As Fischer et al. described it, the partial construction "talks back" to the creator. For example, placing a button in a user interface might suddenly reveal a conflict with screen real estate or user workflow that wasn't apparent in the abstract. This unexpected conflict forces the creator to go back and rewrite the problem specification itself.


Why Fixed Specifications Stifle Innovation

When computational systems try to bypass this co-evolutionary loop by forcing users to commit to rigid, upfront specifications, the results are often uninspired.

Wills et al. observed that when automated design systems do not allow requirements to shift alongside solutions, the resulting outputs tend to be routine and formulaic rather than creative. True innovation requires an opportunistic flexibility where a surprising intermediate solution can redefine what the goal actually is.

Because these fluid domains (such as ethics, language, or creative design) lack rigid, logical rules, traditional rule-based expert systems frequently fail. To bridge this gap, Chaturvedi argues for hybrid cognitive architectures—such as Case-Based Reasoning combined with experiential data—that can navigate the ambiguity of environments where rules are not explicitly stated.


The Modern Frontier: AI Agents and Real-World R&D

This tension between well-defined benchmarks and ill-defined reality remains a core challenge at the frontier of artificial intelligence today.

Modern large language models and autonomous agents perform exceptionally well on curated benchmarks with clean data and explicit objectives. However, as demonstrated by Chan et al. in their evaluation of machine learning agents on MLE-bench, frontier models still struggle when thrown into the messy reality of open-ended research and development.

Real-world machine learning engineering does not begin with a pristine dataset and a clear metric; it begins with vague business goals, uncollected or messy data, and hidden infrastructure constraints. To succeed in these environments, an AI agent cannot just be a passive searcher of a pre-defined optimization space. It must behave like a human designer: actively formulating the problem, probing the environment with trial solutions, listening to how the situation "talks back," and iteratively refining the objective function itself.

Key Foundational Work

Cognitive Processes and III-Defined Problems: A Case Study from Design (1969) — contributed 7 concept elements to this direction.
Cognitive Processes and III-Defined Problems: A Case Study from Design (1969) — contributed 2 concept elements to this direction.
A Model for Functional Reasoning in Design (1971) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_subjective_problem_definition_8add8d9e,
  title = {subjective problem definition},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/subjective-problem-definition/rdr_8add8d9ed5384f81b6058de74d05cbc5},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_subjective_problem_definition_8add8d9e_v1,
  title = {subjective problem definition},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/subjective-problem-definition/rdr_8add8d9ed5384f81b6058de74d05cbc5/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake