Spending Control Effort Where the Batch Is Sensitive: Phase-Dependent Tolerances for Small-Data Bioreactor Control

Video streaming researchers found that viewers do not notice a quality drop equally at every moment. A blurry second during a static shot goes unnoticed, while the same blur during a goal ruins the experience. Systems that learn when viewers are sensitive can save a quarter of the bandwidth at equal satisfaction. This note proposes the same move for bioreactors, the large tanks in which living cells make therapeutic proteins or vesicles. The final quality of a batch is far more sensitive to process deviations in some phases of a run than in others. Today's controllers mostly hold every setpoint to the same tight tolerance from start to finish. We propose to estimate, from only a handful of historical batches, a time-varying profile of how much each phase matters to final quality. We would then use that profile to schedule the controller's tolerances: loosen them where the batch is provably insensitive, saving feed, aeration, and actuator effort, and tighten them where it is sensitive.

The opportunity is real because biomanufacturing has the same structure that made sensitivity-aware streaming work. The objective that matters is expensive to measure and observed only late. In streaming that objective is human opinion collected by crowdsourcing. In manufacturing it is purity and potency, assayed offline after harvest. Meanwhile the controller acts on cheap, frequent proxies. The twist is the data regime. Streaming systems profile each video once and reuse the profile across millions of views. A bioprocess may have three to five prior batches. The profile must therefore carry its own uncertainty, and the controller must relax tolerances only where the evidence for low sensitivity is strong.

Core Claim

A tracking controller that weights deviations by an uncertainty-aware, phase-resolved estimate of terminal-quality sensitivity should reach the terminal quality of uniform tight tracking with substantially less resource use. "Uniform tight tracking" here means the industrial default of fixed setpoints with fixed penalty weights. We also expect this weighted controller to recover most of the gap to a full-knowledge economic optimizer while needing only a small-data statistical model rather than a trusted mechanistic one.

The new ingredient is not sensitivity analysis, and not reinforcement learning for bioprocesses. Both already exist. The new ingredient is to turn a credible-interval sensitivity profile into a time-varying tolerance schedule, and to update it batch by batch. Concretely, the controller minimizes

$$J=\sum_{t} w_t\,\lVert x_t - x^{\mathrm{ref}}_t\rVert^2 + \lambda\, c(u_t), \qquad w_t = \bar S_t^{(1-\alpha)},$$

where $x_t$ and $x^{\mathrm{ref}}_t$ are the measured and reference process states (feed level, dissolved oxygen, temperature) in time window $t$, $u_t$ is the control action, $c(u_t)$ its resource cost, and $\lambda$ trades cost against tracking. The weight $\bar S_t^{(1-\alpha)}$ is the upper $(1-\alpha)$ credible bound on $S_t$. $S_t$ is the share of terminal-quality variance attributable to deviations in window $t$. Using the upper bound makes the controller relax a window only when even a pessimistic estimate says it does not matter.

Research Program

The sensitivity estimator reuses the method of Interpretable biomanufacturing process risk and sensitivity analyses for quality‐by‐design and stability control. That paper builds a Bayesian network over multi-phase process parameters and quality attributes. It attributes output variance with Shapley values, a game-theoretic split of credit among correlated inputs, and quantifies how parameter-estimation uncertainty affects the resulting criticality rankings. The paper's analysis is batch-level and offline, and it ranks parameters rather than time windows. We change three things. We index the network's process nodes by time window, so attribution yields a sensitivity curve over the run rather than a single ranking. We propagate posterior uncertainty into a credible band on that curve. We feed the band into the controller's weights and refit after every new batch, much as Reinforcement Learning under Model Risk for Biomanufacturing Fermentation Control. updates its growth-rate posterior online.

That paper also supplies the main evidence that ignoring parameter uncertainty is costly at three to five batches, which is why the credible bound appears in the weight. Its action is a single harvest-time decision. Our action is continuous regulation throughout the run, with harvest time held fixed or optimized separately.

The control concept comes from SENSEI: Aligning Video Streaming Quality with Dynamic User Sensitivity. SENSEI profiles per-chunk viewer sensitivity offline and reweights existing adaptive bitrate controllers with that profile. It also proactively degrades quality during insensitive chunks to bank buffer for sensitive ones. The bioprocess analogue of banking buffer is deliberately running lean during insensitive windows to preserve substrate or limit impurity accumulation before a sensitive production phase.

SENSEI also gives a negative lesson that constrains our design. Its profiling cost of $31.4 per minute of video makes it uneconomic for low-viewership content. The bioprocess analogue is that every profiling batch costs days and real money. We will therefore report savings as a function of the number of profiling batches, and identify the break-even point.

Comyco: Quality-Aware Adaptive Video Streaming via Imitation Learning shows that optimizing a perceptual quality score rather than raw bitrate removes wasted effort on chunks with negligible perceptual gain. That result motivates using terminal quality rather than raw yield as the target of the sensitivity profile. It also cautions that abrupt transitions, such as the rapid scene cuts that hurt Comyco's policy, are where learned weighting is most fragile. Phase transitions like the switch from growth to production are the bioprocess equivalent, and we will examine them separately.

The deliverables are three. The first is the time-indexed Bayesian network with Shapley sensitivity estimator. The second is the weighted controller, implemented as model predictive control (a controller that repeatedly re-plans actions over a short horizon) on a simple data-fitted model. The third is a set of simulated testbeds with a terminal quality objective.

The testbeds start from existing environments. We extend the stochastic fed-batch fermentation model with an impurity surge used by the model-risk paper, adding continuous feed and temperature actions. We add a terminal economic and quality objective to the continuous stirred tank reactor and crystallization environments of PC-Gym: Benchmark environments for process control problems. We use the two-species chemostat of Deep reinforcement learning for the control of microbial co-cultures in bioreactors as a continuous-process case. Finally, we apply the estimator, without closing the loop, to the historical citric acid fed-batch data analyzed by the Bayesian network paper. There we check whether the estimated sensitivity curve concentrates where that paper found feed rate and residual oil mattered, namely the growth phase.

Extracellular vesicle production is the motivating application, because particle count is a poor proxy for potency there (Advancements in extracellular vesicles biomanufacturing: a comprehensive overview of large-scale production and clinical research). However, the file contains no vesicle-secretion simulator or dataset. Vesicle processes are therefore a stated target for later wet-lab work, not a claimed result.

Baselines

The naive reference is uniform tight tracking. It is a model predictive controller with the same data-fitted model and the same horizon, but with constant weights $w_t$ tuned once by grid search on the profiling batches. This is the industrial default the proposal claims to beat. Beating it means matching its terminal quality within the noise across seeded runs while using measurably less resource. We will report the saving as a percentage at equal quality, the same form SENSEI uses ("26.8% less bandwidth on average" at equal quality). We do not expect that magnitude to transfer.

The strongest existing method is Bayesian model-risk reinforcement learning (Reinforcement Learning under Model Risk for Biomanufacturing Fermentation Control.), extended to our continuous action space with the same posterior and sparse-sampling look-ahead. It is the strongest competitor because it optimizes the true terminal objective directly under small data. That paper reports that with only three historical batches its strategy reached "nearly 90% of the theoretical maximum reward," along with roughly 50% average yield improvement on the industrial data. We will give it the same profiling batches, the same simulator calls per decision, and the same tuning budget as our method.

Beating it would mean one of two outcomes. Either our method reaches comparable percent-of-oracle reward with lower resource use and lower run-to-run variance, or it reaches comparable reward with an order of magnitude less online computation and an interpretable tolerance schedule. If it does neither, that is a negative result we will report.

The ceiling is a full-knowledge nonlinear model predictive controller on the true simulator and objective, the oracle role it plays in PC-Gym. In PC-Gym, soft actor-critic came within an optimality gap of 0.002 of that oracle on the multistage extraction column. We include soft actor-critic trained with PC-Gym's budget as a large-data reference, to show what our small-data methods give up.

Two ablations isolate the key ingredients. The first replaces the credible upper bound with the posterior mean. It tests whether uncertainty-awareness, rather than reweighting alone, prevents quality failures at three batches. The second replaces the fine-grained curve with two or three phase-wise constant weights set from the same Shapley analysis.

For constraint handling we follow PC-Gym's reward-shaping result on the stirred tank. There a shaped agent cut constraint violation probability from 0.0727 to 0.0003. We will report violation probability for every method, so that tolerance relaxation is never bought with unsafe excursions.

Why not just…?

Why not just run economic model predictive control or model-risk reinforcement learning on the terminal objective, which already accounts for when deviations matter? This is the real threat. With an accurate model, an optimizer on the true objective implicitly contains the sensitivity schedule, and our weighting adds nothing. Our bet is about the small-data regime, in which the terminal-quality model is the least trustworthy part of the system. A variance-attribution estimate with credible bands may be more robust there than a full plant-plus-quality model optimized end to end.

The experiment settles this directly. Using the extended model-risk baseline, we sweep the number of profiling batches from three to twenty. If the baseline matches us at every batch count in percent-of-oracle and resource use, the contribution reduces to interpretability and computational cheapness, and we will say so.

Why not just use the phase-specific setpoints process engineers already write, such as separate growth and production phases? Coarse phasing captures part of the effect. The Bayesian network paper's finding that growth-phase parameters dominate the citric acid outputs suggests much of the sensitivity may already sit at phase granularity. The phase-wise ablation is exactly this alternative given the same data. If fine-grained weights do not beat it by more than seed variance, the right conclusion is that a two-phase schedule, set by Shapley attribution rather than by intuition, is enough. That would still be a useful and deployable result.

Why not just retune a PI controller per phase? Proportional-integral control (feedback on the current error and its accumulated history) is cheap and validated. However, the chemostat study found it deteriorates once sensor sampling is sparser than five minutes, where a learned controller had consistently lower squared error. More fundamentally, retuning gains changes how hard the controller pushes, not what it is allowed to ignore. We include per-phase-tuned PI as an additional reference on the chemostat and fermentation testbeds under sparse sampling.

Why not just maximize raw yield and let downstream purification clean up? The model-risk paper shows yield-only harvesting drives the process into an impurity surge and raises batch-to-batch variability. Comyco shows the streaming analogue: bitrate-maximizing policies waste resources on chunks with no perceptual gain. We will report a yield-only controller to show the gap in the purity-penalized terminal objective.

Proposed Experiments

The decisive experiment runs on the extended fermentation simulator. We vary the number of profiling batches over three, five, ten, and twenty, with fifty seeded test batches per setting. For each method we report terminal objective as a percentage of the full-knowledge oracle, total resource use (feed, aeration energy, actuator travel), constraint violation probability, and coefficient of variation of terminal quality.

The proposal succeeds if it beats uniform tracking on resource use at equal quality, and if the credible-bound weights beat mean weights on violation rate at three batches. Against model-risk reinforcement learning, both outcomes are informative and will be reported as such.

The same protocol is repeated on the PC-Gym stirred tank and crystallizer with terminal objectives. Crystallization is where PC-Gym reports all reinforcement learning agents doing poorly against the oracle, so it tests whether reweighting helps where learned policies struggle. The protocol is also repeated on the chemostat under five-, ten-, and twenty-minute sampling.

A diagnostic check plants a known sensitivity structure in the simulator, with quality affected only by deviations in a chosen window. We then measure how many batches the estimator needs before its credible band excludes zero sensitivity outside that window. This also gives the profiling break-even analogous to SENSEI's cost argument.

Finally, on the citric acid historical data, we test whether the time-resolved sensitivity curve reproduces the Bayesian network paper's growth-phase findings for feed rate and residual oil. This is a sanity check that the estimator recovers known structure from real data before any wet-lab vesicle trial is attempted.