Engineers who tune software by hand, such as a Spark job's memory and parallelism settings or a recommender's training options, usually know which settings matter together and which do not, because they have read the code. Automatic tuners generally do not use this knowledge. They treat all settings, here called "knobs," as one entangled black box. This note proposes reading that structure out of the program mechanically. The tool is backward program slicing, a classical analysis that starts from a chosen output and keeps only the statements that can affect it. We run it from each cost-bearing part of the program back to the places where knobs are read. The result shows which knobs never touch the objective, which knobs only matter when another knob selects a particular code path, and which groups of knobs act on disjoint parts of the computation. That structure then fixes the shape of a Bayesian-optimization search model before any tuning trial is spent.
The bet is simple. A slice is free to compute relative to a Spark run, and it is conservative: it never claims independence that the code does not support. It should therefore give a better starting structure for the search than one learned from a handful of expensive trials. It should also give a better starting structure than a whole-program embedding that mixes logging, I/O, and setup code into the signal. The idea also suggests a new way to transfer knowledge between programs. Instead of judging two programs similar as wholes, we reuse past observations for one knob group whenever the two programs share the slice that the group acts on.
Core Claim
Backward slices rooted at the cost-bearing operations of a program induce three facts about the tuning objective. The first is the set of knobs that cannot affect it. The second is a conditional structure, coming from control dependence: knob $b$ is only read on a branch selected by knob $a$. The third is a partition of the remaining knobs into groups $G_1,\dots,G_m$ whose slices do not intersect. For objectives that add up over program stages, such as total execution time, this partition justifies an additive surrogate model:
$$k(x,x') = \sum_{j=1}^{m} k_j\big(x_{G_j}, x'_{G_j}\big),$$
Here $x$ is a knob configuration, $x_{G_j}$ is its restriction to group $G_j$, and each $k_j$ is an ordinary Gaussian-process kernel over that group. Over-approximate slices only merge groups. Analysis imprecision therefore costs search efficiency, not correctness.
We claim that this slice-derived surrogate reaches a given configuration quality in fewer trials than both unstructured Bayesian optimization and code-embedding transfer. We also claim that the advantage tracks how modular the program is. It should be large for multi-stage dataflow jobs and near zero for dense numerical solvers, where every knob influences every output value.
Relation to Prior Work
The closest work is Code-Aware Cross-Program Transfer Hyperparameter Optimization (CaTHPO). It embeds a program's abstract syntax tree (the parsed tree form of its source code) with a Transformer and an LSTM, then uses whole-program similarity to weight past runs in a Gaussian-process surrogate and to condition a learned acquisition policy. It motivates this proposal, because it shows that program code beats dataset statistics as a transfer signal. Our method differs in kind, not just in input. CaTHPO segments the syntax tree at random and never computes dependence. We use dependence to fix the surrogate's structure and to transfer at the level of slices rather than whole programs.
The slicing method itself comes from classical work. The Semantics of Program Slicing and Program Integration gives the guarantee that a slice reproduces the values at its criterion. That guarantee is what licenses reading non-membership in a slice as non-influence. Dynamic slicing, as studied in Cost effective dynamic program slicing and Locating Faults with Program Slicing: An Empirical Analysis, follows dependence along concrete executions. We need it because Spark knobs are read inside framework code, where static alias and dispatch analysis inflates slices. Slicing probabilistic programs shows that slicing can make a downstream statistical procedure faster by removing irrelevant computation. That work prunes the model being inferred. We prune and factor the search space of the procedure that tunes the program.
The tuning-free inverse-problem literature constrains where the idea can help. Tuning-Free Plug-and-Play Hyperspectral Image Deconvolution With Deep Priors sets the penalty parameter by checking that the reconstruction residual looks like white noise. Improving Diffusion Models for Inverse Problems Using Optimal Posterior Covariance removes guidance-scale tuning by estimating covariance. Both show that some knobs in iterative image-reconstruction solvers should be derived rather than searched. In such dense iterations every knob reaches every pixel, so slicing yields no factorization there. We treat these solvers as a boundary case, not a target.
Research Program
The work has three parts. The first is a slicer for tuned programs. A few profiling runs at default settings are instrumented. Stage boundaries, such as Spark stages or recommender training-loop phases, define the slicing criteria, which are each stage's cost-bearing operations. Dynamic dependence tracing records every configuration read that falls inside each stage's backward slice, together with the branch conditions guarding it. From this we extract three outputs: the dead knobs, a conditional activation tree, and the knob-to-stage incidence that defines the groups $G_j$. Knobs reaching several stages land in the same group. We will also measure how often imprecision collapses everything into a single group, since that failure mode is named in the slicing literature.
The second part is the search itself: Bayesian optimization with the additive, conditional kernel above. The same acquisition function is used everywhere, so that any gain comes from structure alone.
The third part is slice-level transfer. Each group's observations are keyed by a canonical hash of its slice, meaning the normalized dependence subgraph. When a new program shares that slice, those observations become a prior for that component only.
Baselines
The naive reference is standard Bayesian optimization with a single joint kernel over all knobs and no prior data. CaTHPO's headline numbers are reported against exactly this baseline. The second baseline is random search at equal budget, which shows whether any structure helps at all.
The strongest existing competitor is CaTHPO, run on its own Spark Tuning Suite (fifteen knobs, eight applications) and Recommender System suite (eight models, NDCG@10). It reports an average 24% execution-time reduction over standard Bayesian optimization on Spark, and NDCG@10 gains of 10% to 96% over various competitors on the recommender suite. Absolute per-application times are not reported in the file, so we will re-run CaTHPO from its released setup if available, or from a faithful reimplementation. Both methods get identical leave-one-program-out transfer pools, trial budgets, and hardware. Beating CaTHPO means a larger execution-time reduction over standard Bayesian optimization at equal budget, or matching its final quality in at most half the trials on held-out programs.
Two ablations isolate the slice. The first is a syntactic-reachability grouping: knobs are grouped by which functions textually mention them, with no dependence analysis. The second is a combination of CaTHPO's embedding with our additive kernel. This separates the value of the structure from the value of slice-level transfer.
Why not just…?
Why not just learn the interaction structure from the trials themselves? Sensitivity or additive-structure discovery from data is the real threat. With enough evaluations it should recover the same groups without any program analysis. Its weakness is the low-budget regime of tens of Spark runs, where estimated interactions are noisy. We will compare data-learned and slice-derived structure at budgets of 10, 20, 50 and 100 trials. If the learned structure matches ours by roughly 20 trials, the contribution shrinks to a warm start, and we will report it that way.
Why not just give the code to a large language model and ask which knobs interact? This is cheap and plausible. The file contains no reported result on Lacuna for language-model knob-dependence extraction. Using DI-BENCH for dependency inference suggests models can partly read dependencies from repositories, but it gives no soundness guarantee. We will add this as a third structure source and score each source's groups against ground-truth interactions measured by exhaustive two-knob sweeps on a subset of applications.
Why not just rely on CaTHPO's whole-program embedding? Its random segmentation keeps boilerplate in the representation. More importantly, it can only say that two programs are similar overall. It cannot say that their shuffle stages are identical while their compute stages differ. The slice-transfer arm tests whether that finer granularity matters, using held-out programs that share only some stages with the transfer pool.
Proposed Experiments
The main experiment uses the Spark suite with leave-one-program-out transfer. We report execution time against trial count for every method, along with the number of trials needed to reach CaTHPO's final quality. The second experiment uses the recommender suite. NDCG@10 is not additive over stages, so we expect gains mostly from dead-knob removal and conditional structure, and we will check whether the additive kernel hurts there.
The decisive check is the correlation, across applications, between slice modularity (number of groups relative to the number of knobs) and the trial savings our method achieves. A positive correlation supports the mechanism. Gains that are uncorrelated with modularity would suggest the improvement comes from something else, such as dead-knob pruning alone.
The boundary experiment tunes the full flag set of a diffusion inverse solver on FFHQ inpainting. We count the trials needed to reach the published DPS LPIPS of 0.1323 reported by Improving Diffusion Models for Inverse Problems Using Optimal Posterior Covariance. We predict little benefit beyond pruning flags that the chosen sampler never reads. Confirming this prediction delimits the method honestly.
References
- Code-Aware Cross-Program Transfer Hyperparameter Optimization
- The Semantics of Program Slicing and Program Integration
- Cost effective dynamic program slicing
- Locating faults with program slicing: an empirical analysis
- Slicing probabilistic programs
- Tuning-Free Plug-and-Play Hyperspectral Image Deconvolution With Deep Priors
- Improving Diffusion Models for Inverse Problems Using Optimal Posterior Covariance
- DI-BENCH: Benchmarking Large Language Models on Dependency Inference with Testable Repositories at Scale