Declared Decisions as the Unit of Search in Simulator-Driven Cache Policy Discovery

Language-model agents can now invent new hardware mechanisms, such as the rules a CPU cache uses to decide which line to evict. They do this by writing code and scoring it in a cycle-accurate simulator, a program that models the processor clock tick by clock tick. Each score is slow: a full run takes many hours, and the agent writes free-form C++ that the search loop treats as an opaque string. Two costs follow. First, almost every candidate has to be simulated before anything is known about it. Second, the agent is free to exploit simulator bugs. One discovered policy silently dropped cache writes and looked like a large speedup. We propose to change what the agent produces. Instead of arbitrary code, it fills in a typed architectural decision record. This is a short, structured declaration of what the policy reads, how it splits cache sets into groups, which priority it gives to each kind of line, and in what order it evicts. The record is then compiled into executable policy code.

Because every candidate is now a point in an explicit decision space, three things become cheap. Before simulation, we can reject candidates that break hardware storage budgets or structural rules. We can attach runtime checks to each candidate that follow from what it declared it would do. And we can fit a fast predictor over decision records that chooses which candidates deserve simulator time. The question we test is whether this constraint-first structure, borrowed from architectural decision modeling in systems engineering, reaches the same cache performance as free-form agentic search with a fraction of the cycle-accurate simulation budget and with no undetected simulator exploits.

Core Claim

The decision-modeling literature argues that architecture selection should start from a formal map of constraints and decisions, not from browsing a catalog of designs. Architecture Design Space Generation via Decision Pattern-Guided Department of Defense Architecture Framework Modeling formalizes six reusable decision patterns: Downselecting, Partitioning, Assigning, Permuting, Connecting, and their Combination. In a defense system-of-systems case, interdependencies among decisions cut the space to 34 viable alternatives. A Decision Model for Selecting Patterns and Strategies to Decompose Applications into Microservices shows the same effect for software: putting constraints first blocks unviable patterns early. Both works stop at qualitative evaluation, with no link to a performance simulator. That limitation motivates our work.

On the hardware side, ArchAgent: Towards Agentic Architecture Discovery couples an evolutionary language-model loop to the ChampSim simulator. It manages cost with a two-stage cascade that screens on 100M instructions and confirms on 1B, and it reports the simulator escape described above.

Our claim is specific. When candidates are expressed as instances of these decision patterns rather than as free code, a single schema produces three things: a pre-simulation feasibility filter, candidate-specific runtime invariants, and a shared feature space for a cheap ranking model. Together these cut the number of cycle-accurate simulations needed to reach a target IPC (instructions per cycle) by a large factor, and they eliminate escapes, without lowering the best IPC reached. ArchAgent searches free code with no feasibility model. The decision-modeling works build feasibility models but never connect them to a simulator. We do both in one loop and measure what the structure buys.

Research Program

The first deliverable is a decision schema for cache replacement policies, written once by hand. This is an honest concession to the open problem that decision matrices are built manually in the DoDAF work. In our setting the hand-built cost is paid once per mechanism class, not once per design. The schema maps each pattern to replacement-policy choices:

  • Downselecting chooses which signals a reuse predictor may read, such as program counter bits, address bits, or a recency history.
  • Partitioning splits cache sets into leader and follower groups for set dueling. It must be mutually exclusive and collectively exhaustive.
  • Assigning maps predicted line classes to insertion priorities, including bypass.
  • Permuting fixes the eviction order among candidates.
  • Connecting decides which predictor tables are shared across cores.

The agent proposes a record. A deterministic compiler, in the spirit of the abstract-syntax-tree templates that let SymRTLO keep language-model rewrites inside a verified skeleton, turns the record into ChampSim C++. Each pattern also has an escape hatch: a small, size-limited code slot where the agent may write a custom scoring function. We measure how often the agent uses it, since that tells us how much expressiveness the schema costs.

The second deliverable is what each record compiles into besides code:

  • A storage and logic-depth formula. Records that exceed a fixed hardware budget are rejected before any simulation.
  • Runtime invariants derived from the declaration. If a record assigns bypass to any class that can contain dirty lines, the compiler inserts a check that every bypassed dirty line produces exactly one lower-level write. If a record declares a partition, the compiler checks that each set falls into exactly one group. These checks are compiled into a build of the simulator whose assertions cannot be optimized away. That optimization is exactly what let Policy12 pass unnoticed in ArchAgent.

The third deliverable is the budget allocator. All candidates share the decision schema as a feature space. We fit a small ranking model on every 100M-instruction result gathered so far. It is a gradient-boosted tree over decision features plus per-workload miss statistics, retrained online. Each round, only the top fraction of new records go to the 100M screen, and only the top fraction of those go to the 1B confirmation. This keeps ArchAgent's cascade intact but inserts a learned zero-cost stage in front of it, and it makes that stage possible because candidates are structured.

The structure also clarifies a tension in the literature. Multi-Agent Reinforcement Learning for Microprocessor Design Space Exploration found that memory-controller knobs are weakly correlated and can be tuned independently. Decision patterns are coupled by design: an Assigning choice that includes bypass constrains which Downselecting signals are useful. We will measure the interaction strength between pattern choices in the 100M results. That tells us whether a ranking model with pairwise interactions is needed or whether an additive one suffices.

A secondary, clearly separated thread asks whether physical realizability should also gate simulation. Because records compile through templates, we can also emit small Verilog modules for the policy's update logic and score them with MasterRTL. MasterRTL predicts area and timing from register-transfer-level code, before logic synthesis, in under 0.1 seconds. It reports Pearson correlations of 0.98 for area and 0.96 for total negative slack on its 90-design suite. That suite contains processors and accelerators, not small replacement-logic blocks, so we treat transfer as untested. We check it against Yosys synthesis on a random sample of records before trusting it as a filter.

Baselines

The naive reference point is fixed human policies. LRU, SHiP, and Mockingjay are simulated on the same traces and in the same ChampSim configuration. They bound what any search must beat and are the denominators for all reported speedups.

The strongest existing method is the free-form evolutionary agent of ArchAgent, with its 100M-to-1B cascade. Its reported numbers are an 8.2% IPC improvement over LRU and a 5.3% improvement over SHiP on Google Workload Traces (found in 2 days), and a 0.9% IPC speedup over Mockingjay on a 19-workload memory-intensive SPEC CPU 2006 subset (found in 18 days). We cannot rerun Gemini through AlphaEvolve as-is. We will therefore reimplement the loop with one fixed language model, prompt template, and evolutionary database shared by every agentic condition. Each condition gets the same number of model calls and the same count of 100M and 1B simulations.

Beating this baseline means one of two things:

  • reaching the reimplemented free-form agent's final best 1B IPC with at most half of its 100M simulations, or
  • exceeding its best IPC at equal budget, with zero confirmed escapes among our reported winners.

We will report our reimplementation's numbers alongside ArchAgent's published ones, not substitute one for the other.

The ablation that isolates the key ingredient keeps everything except the structure. It is the free-form agent equipped with our generic conservation assertions and a ranking model trained on code embeddings of its candidates. If this matches our method, the gain comes from having a surrogate, not from declared decisions.

A second ablation uses decision records with the feasibility filter and invariants but no ranking model. This separates pruning from prediction.

The parametric baseline is Bayesian optimization over the decision schema's categorical and integer fields, with no language model, under the same simulation budget. AutoDSE reports outperforming Bayesian optimization by 3.6x to 17.9x under equal time. That was in FPGA high-level synthesis, so we do not assume it transfers. We found no reported result on Lacuna for Bayesian optimization or random search over replacement policies in ChampSim, which is why we run both ourselves.

Why not just…?

Why not just add conservation assertions to ChampSim and keep free-form search? This is the most serious threat to the escape part of our claim. A generic invariant like "dirty lines in equal writes out" would have caught Policy12. We include exactly this as the main ablation and expect it to close most of the escape gap. What generic checks cannot express is a declared intent that the behavior contradicts, such as a policy claiming partitioned set dueling while actually updating all sets. We will count escapes of that kind found by audit, by differential simulation against a trusted reference cache model, in both conditions. If none occur, we will say that escape prevention needs only generic checks, and the contribution rests on the budget results.

Why not just run constrained Bayesian optimization over a parameterized policy family? If the schema already defines the space, an optimizer over its fields might do as well without a language model. It cannot propose new scoring functions in the escape-hatch slot. Policies like ArchAgent's Policy31, which combines an insertion quality predictor with a separate tracking mechanism, came from the agent writing new code. The Bayesian optimization baseline runs at equal budget. If it matches our best IPC, the language model is unnecessary given the schema. That would still be a useful finding, because it would make constraint-first modeling the active ingredient.

Why not just fit a surrogate on free-form code, or lengthen the instruction cascade? A ranking model over code embeddings needs no schema, and a three-stage cascade (10M, 100M, 1B) is trivial to add. Both are run under the same simulation budget. The embedding surrogate is the key ablation. The extra cascade stage is applied to every condition, so any advantage we report holds on top of it. If code embeddings predict 100M IPC as well as decision features do, measured by rank correlation on held-out candidates, our structural argument fails on that axis.

Why not tune each decision independently with per-knob agents, as in DRAM-Gym? The independence result of the multi-agent study holds for memory-controller knobs. We measure interaction effects between patterns directly. If they are weak, an additive model and independent tuning are adequate, and we will adopt the simpler model.

Proposed Experiments

Workloads are the SPEC CPU 2006 memory-intensive subset in single-core, prefetch-enabled ChampSim and the multi-core Google Workload Traces, matching ArchAgent. Within each suite, we split traces into search and held-out sets, so that winners are judged on traces never used during search.

The primary metric is the curve of best held-out 1B IPC, relative to SHiP and Mockingjay, against the cumulative number of cycle-accurate simulated instructions. From that curve we report the simulation budget needed to reach the free-form agent's final IPC. Secondary metrics are:

  • the fraction of proposals rejected before simulation, and how many of those, when simulated anyway on a 5% audit sample, would have entered the top decile;
  • confirmed escape counts per thousand candidates;
  • the ranking model's rank correlation with 100M results;
  • how often the agent uses the code escape hatch, and the IPC of winners that used it versus those that did not.

The decisive check is a single comparison at matched budget: our method against free-form search with generic assertions and an embedding surrogate. Three outcomes are possible:

  • If our method reaches the target IPC with clearly fewer simulations, and the audit shows its rejected candidates were rarely good ones, declared decisions earn their place.
  • If both reach the target with similar budgets, the benefit comes from the surrogate, and we report that.
  • If our winners are consistently worse because the schema excludes good designs, as shown by an outsized share of top free-form policies that cannot be expressed in the schema, then constraint-first structure costs more than it saves for mechanism discovery.

The realizability thread is reported separately. We give the MasterRTL-versus-Yosys correlation on generated policy modules, and, only if that correlation is high, the number of IPC winners that a realizability gate would have removed.