How Much of the Gain in Learned Heat-Recovery Control Survives a Physics-Concept Bottleneck?

Learned controllers now beat hand-written rules at running data-center cooling loops that send waste heat to reuse. On a digital twin of the Frontier supercomputer's liquid cooling plant, reinforcement learning (RL) agents keep server blades inside their temperature band far more often than the industry rulebook does. They also cut cooling-tower power by about a fifth when a heat recovery unit diverts warm coolant for reuse (LC-Opt). Plant operators still run the rulebook. They cannot audit a neural policy, and the explanations bolted on afterwards are either an 830-node decision tree or a language model's free-text narrative, which nobody can check against an energy balance.

We propose to build and measure a controller that cannot hide its reasoning. It first computes a small set of named physical quantities from plant sensors: heat currently recovered, coolant approach temperature at each exchanger, cooling-tower approach to wet-bulb temperature, and remaining thermal margin per blade group. It then sets every pump, valve, and fan using a short gain-scheduled program over only those quantities. The scientific question is not whether this controller is "interpretable enough." The question is what fraction of the black-box RL improvement over the rulebook is actually recovered by such a program, and how much of the gap was really caused by poor tuning of the rulebook rather than by missing neural capacity. That decomposition is what an operator deciding whether to trust learned control needs, and no paper in this file reports it.

Core Claim

We claim that on liquid cooling with heat recovery, most of the black-box RL advantage passes through a handful of thermodynamic concepts. The claim has two parts. First, a policy whose only inputs are those concepts and whose decision rule is a sparse piecewise-linear program with a few operating regimes will close most of the gap between ASHRAE Guideline 36 and PPO. Second, a meaningful share of that gap closes from tuning the rulebook's own parameters alone. If the claim fails, the experiments still show where the residual advantage of opaque control lives: in fast transients, in cross-loop coupling, or in information the concepts do not carry. That result is equally useful.

This differs from the closest work in a specific way. LC-Opt trains an unconstrained PPO policy and only afterwards compresses it into a tree or a fine-tuned Qwen model. Its explanations are therefore descriptions of a different model. In our controller the program over concepts is the policy, so the explanation is exact by construction. Our concepts are physical quantities with known units that can be checked against the simulator's own energy accounting, rather than learned splits on raw sensor values.

Research Program

Three existing lines of work supply the method, and each is used for a different role.

Concept bottleneck models — networks forced to predict human-named intermediate quantities and then decide only from those (Concept Bottleneck Models) — give us the architecture. That work and its descendants define concepts as visual or textual attributes that must be annotated. We instead compute most concepts directly from the twin's sensors with first-law balances. Mass flow times specific heat times temperature difference gives heat recovered. Outlet temperature minus inlet coolant temperature gives exchanger approach. Annotation therefore disappears. The one concept that needs learning is a short-horizon forecast of blade thermal margin, and we check it against simulator ground truth.

Programmatically interpretable RL (Programmatically Interpretable Reinforcement Learning) gives us the training recipe. That method imitates a neural oracle with DAgger, a procedure that repeatedly queries the oracle on states the imitating program actually visits. It fits a human-written program sketch, tunes constants by Bayesian optimization, and in TORCS produced far smoother steering than the neural agent. Its main limitation is that a human hand-writes the sketch. In our setting the sketch writes itself from domain practice, because industrial loops are already PI-style controllers. Our sketch is a regime selector, a shallow tree of depth at most three over concepts, feeding per-regime affine control laws per actuator. Search is over regime thresholds and gains. The PPO oracle from LC-Opt serves as teacher, followed by direct policy-gradient fine-tuning of the program's continuous parameters in the environment.

Prototype actors (This State Looks Like That) are the main alternative architecture. They act by similarity to a few stored archetypal states. ProtoSAC matched or beat Soft Actor-Critic in 6 of 8 control tasks but lost on HalfCheetah, which needs fast torque modulation. That limitation constrains our design. Heat recovery mixes slow exchanger thermal inertia with fast pump and bypass-valve response, so we include a ProtoSAC actor over the same concepts as a competitor rather than assume it works.

The deliverable is an accuracy-versus-program-size curve. We sweep the number of regimes and the number of concepts each control law may read, and plot blade-temperature compliance, cooling-tower power, and heat recovered against total program size in readable terms. Along this curve we place the tuned rulebook, the LC-Opt-style distilled tree, ProtoSAC, and black-box PPO. We also run a concept-ablation study. Each concept is replaced by its running mean and we measure the performance drop. This tells an operator which physical quantity the controller depends on, a statement neither a SHAP plot nor a language model rationale can make faithfully.

Baselines

The naive reference point is ASHRAE Guideline 36 at its default settings, run in LC-Opt as the authors provide it. LC-Opt reports blade-group temperature compliance of 76.92% for this baseline.

The second baseline is what a skeptical engineer would actually do: Guideline 36 with its setpoints, deadbands, and PI gains tuned by Bayesian optimization in the same environment. It receives the same number of environment episodes as our program search and the same weather and workload traces. We found no reported result on Lacuna for a tuned Guideline 36 on LC-Opt, so we must produce this number ourselves. It defines the tuning gap.

The strongest existing method is LC-Opt's best black-box policy: multi-agent PPO with centralized action execution and a multi-head policy, with the heat recovery unit active (their Case 7). LC-Opt reports 95.63% best-RL compliance, an approximately 21% cooling-tower power reduction with the heat recovery unit, and a two-day carbon figure of 19.22 against 25.24 for the baseline (reported verbatim as "Tonnes CO2/kWh"). We retrain it with the authors' configuration so that seeds and evaluation traces match ours, rather than comparing against printed numbers. Beating it is not the goal. Our controller counts as a success if it recovers at least 80% of PPO's compliance improvement over default Guideline 36 and 80% of its cooling-tower power reduction. It must also clearly exceed the tuned rulebook, at a program size an engineer can read in one sitting.

The interpretable competitor is LC-Opt's distilled decision tree, 830 nodes distilled from a 9,606-parameter PPO oracle. LC-Opt reports the tree's size; we found no reported compliance or power result for it on Lacuna, so we regenerate it from our retrained oracle. Beating it means equal or better control at an order of magnitude fewer decision points. ProtoSAC over the same concept inputs is the second interpretable competitor. We found no reported result on Lacuna for ProtoSAC in thermal control.

Two ablations isolate the key ingredient. The first gives the identical program sketch raw sensor inputs instead of concepts, which tests whether the physics features matter. The second gives black-box PPO the concepts as extra inputs, which tests whether the concepts merely help learning or genuinely carry the decision.

Why not just…?

Why not just tune the existing rulebook harder? This is the most serious threat, and we treat it as a hypothesis rather than an objection. LC-Opt compares against Guideline 36 as written, so part of the 76.92% versus 95.63% gap may be untuned setpoints. Our tuned Guideline 36 baseline measures that directly. If tuning alone closes most of the gap, the honest conclusion is that operators do not need learned control here. We would then report the result and redirect the concept program toward the residual, for example heat-recovery dispatch, which Guideline 36 does not natively schedule.

Why not just distill the black box afterwards, as LC-Opt does? A distilled tree or language model explains a surrogate, not the deployed policy, unless the surrogate is deployed. An 830-node tree over raw sensors is hard to audit during a transient. Our comparison runs both at matched evaluation, reports tree size against program size, and asks whether the tree's splits correspond to any physical quantity. The ablation that hands PPO the concepts and then distills it is the fairest version of this alternative, and it is in the plan.

Why not just use the black box and attach saliency or SHAP explanations? Post-hoc attributions describe sensitivity, not the decision path, and the ProtoSAC authors argue they are often unfaithful to the model's logic. More concretely, attributions cannot answer the operator's real question: what happens to the valve if recovered heat drops by 10%. Our program answers that by reading off one gain. We also verify that the answer holds in the twin.

Why not just use model predictive control? Model predictive control, which repeatedly solves a short-horizon optimization over a plant model, is transparent in its objective but opaque in its moment-to-moment trade-offs, and it is costly to run on a full Modelica plant. The offline-RL server-room study reports its learned controller beating MPC, especially under high load (Data Center Cooling System Optimization Using Offline Reinforcement Learning). That study used air-cooled rooms without heat recovery, so the evidence is indirect. We include a linearized MPC with the same concept-level model as a secondary baseline if the compute budget allows, and report it honestly either way.

Proposed Experiments and Decisive Checks

All main experiments run in the LC-Opt environment with the heat recovery unit enabled, over the benchmark's weather and workload traces, with five seeds per method and identical evaluation windows. Metrics are blade-group compliance, cooling-tower and pump power, heat recovered, the carbon figure LC-Opt reports, actuator smoothness (standard deviation of command changes, following the PIRL steering measure), and program size.

Three checks decide the claim. The first is the gap decomposition: the fractions of the PPO-over-default improvement achieved by tuned Guideline 36, by the concept program, and by the raw-sensor program. The claim fails if the concept program recovers no more than the tuned rulebook. The second is a fast-transient stress test. Sharp workload steps reproduce the HalfCheetah failure mode, where similarity-based and regime-based controllers lag. Here we report whether the concept program, ProtoSAC, or neither keeps blade margin, and which concept the failure traces to. The third is a robustness test motivated by PIRL's sensor-dropout result. We drop or bias 30–70% of sensors and shift the weather to traces unseen in training. We expect the concept program to degrade gracefully, because each concept is a physical balance that can be computed redundantly. We expect raw-input PPO to degrade sharply. If PPO degrades no worse, the robustness argument for the bottleneck is withdrawn.

Finally, we check that the concepts are physically sound. Computed and forecast concepts are compared with the twin's internal heat flows. The program is accepted only if its concept errors stay within instrument-level tolerance. That condition is what lets an operator read the program's decision as a statement about the plant rather than about a network.