Many scientific measurements are bought in time. A telescope stares at a source for some number of days, a clinician monitors a tear film for twenty seconds, a sensor dwells on a tumbling rocket body. If the window is shorter than the timescale you care about, you learn almost nothing about it. Worse, slowly varying "red" noise inside a short window readily produces peaks that look like oscillations. The TESS blazar literature shows both failures. Pininti et al. found no persistent periodicities or spectral breaks within 21–28 day sectors (Exploring short-term optical variability of blazars using TESS). He et al. found 3.1- and 3.7-day oscillations in PKS 1440-389 at more than 4σ, and those oscillations came and went within a sector (Optical Quasi-Periodic Oscillation of Blazar PKS 1440-389 in the TESS Light Curve). Today, whether a source is watched long enough is fixed in advance by a survey schedule.
We propose to treat an observing campaign as a sequential decision problem. With a fixed total time, the observer repeatedly chooses to keep watching the current target, switch to another, or stop and declare a detection or a non-detection. The claim is that the right choice depends on more than the current posterior evidence and the time remaining. It also depends on how the past observing time was arranged, above all how many uninterrupted cycles of a candidate period have been covered. This is because evidence for periodicity grows superadditively with contiguous duration: a window shorter than one period is worth almost nothing, and two broken half-windows are worth far less than one whole window. A planner that looks only one step ahead will therefore under-commit to long stares. A planner that tracks a small statistic of the observing history should recover most of the value of planning over the full belief.
Core Claim
Consider a campaign over targets $i = 1,\dots,N$ with a hard total time budget $T$. The planner chooses dwell segments and, at the end, declares $D_i \in \{0,1\}$ for each target. The campaign is scored once, as a single trial:
$$U = \sum_i \big( D_i Q_i - \lambda D_i (1-Q_i) \big) \quad \text{subject to} \quad \sum_{k \le t} \tau_k + s_k \le T \ \text{ for every step } t.$$
Here $Q_i \in \{0,1\}$ indicates whether a true oscillation is present, $\lambda$ is the penalty for a false claim, $\tau_k$ is the length of the $k$-th dwell, and $s_k$ is the random time lost to slews and data gaps.
The utility is a threshold function of the final evidence, and the budget is consumed stochastically. Together these break the Markov property in the way two lines of work have studied. The first is anytime-constrained reinforcement learning, which requires the running cost to stay under budget on every trajectory (Anytime-Constrained Reinforcement Learning). The second is single-trial general-utility planning, where a nonlinear objective of one realized trajectory forces history-dependent policies (Solving General-Utility Markov Decision Processes in the Single-Trial Regime with Online Planning).
Our specific hypothesis concerns what to summarize from the history. We propose a compact state: remaining time, and for each target its current periodogram significance against a simulated red-noise null, the longest contiguous span covered measured in cycles of the leading candidate period, and the number of gaps. We claim that planning over this state matches a full-belief planner and clearly beats myopic and fixed-schedule policies. We also claim the advantage should grow with the ratio of candidate period to minimum dwell length.
Relation to Prior Work
The closest existing systems are learned observation-cost policies, and they supply the comparison point rather than the method. OCMDP decouples a policy that decides what to observe from a control policy, links them through a learned belief encoder, and optimizes expected return minus sensing cost (OCMDP: Observation-Constrained Markov Decision Process). Related work learns when to request a measurement (Dynamic Observation Policies in Observation Cost-Sensitive Reinforcement Learning; Active Measure Reinforcement Learning for Observation Cost Minimization). In all of these, observations are instantaneous queries with additive cost, and the cost appears only as an expectation. The value of a query does not depend on whether it extends an unbroken window, and the budget is never a hard limit on a single run. Both properties are the substance of our problem.
The history-dependent decision literature supplies the methods. From McMahan and Zhu we reuse two techniques: augmenting the state with discretized remaining budget, and their optimistic cost discretization, which they showed solves a 100-step synthetic knapsack-style problem in 2 seconds at $\epsilon = 0.1$ (Anytime-Constrained Reinforcement Learning). From Santos et al. we reuse online Monte Carlo tree search over an augmented "history" state; on FrozenLake entropy exploration it scored 0.40 against 0.48 for an expected-objective solver, lower being better (Solving General-Utility Markov Decision Processes in the Single-Trial Regime with Online Planning).
Santos et al. also constrain the design. Their running occupancy statistic scales with the number of state–action pairs, so we cannot carry a full history and must find a compact one. Patil et al. learn a recurrent compressed history (an "approximate information state") for MDPs whose state abstraction loses information (On learning history-based policies for controlling Markov decision processes). We use their approach as the learned alternative to our hand-designed statistic.
The astronomy papers motivate the problem and calibrate the simulator. Pininti et al. supply red-noise power-spectrum slopes of 1.7–3.2 and the finding that short windows yield bimodal flux distributions (Exploring short-term optical variability of blazars using TESS). He et al. supply realistic transient oscillations at a few days' period that are visible in a 22.8-day sector (Optical Quasi-Periodic Oscillation of Blazar PKS 1440-389 in the TESS Light Curve).
Research Program
We will build a campaign simulator. Each target's light curve is drawn from power-law red noise with slope uniform in [1.7, 3.2]. With some probability, a damped, time-limited oscillation with a 2–8 day period is injected. The simulator also adds TESS-like downlink gaps and random slew losses. The observer reveals the light curve only in segments it pays for.
Evidence is scored with a Lomb-Scargle periodogram, a standard tool for finding periodicities in unevenly sampled data. Significance is calibrated against Monte Carlo red-noise surrogates following the power-spectral-response fitting practice the blazar papers use. Doing this once per step is expensive. We will therefore precompute a lookup from (span in cycles, gaps, spectral slope) to the null distribution of the peak.
The planner is Monte Carlo tree search over the compact state, with remaining time discretized in the McMahan–Zhu manner. We will then analyze when the compact state loses information relative to a particle-filter belief. Our expectation is that losses concentrate where an oscillation switches off partway through a long window, the non-stationarity He et al. observed.
The second domain tests generality. It uses the RoBo6 rocket-body light curves (LCDC: Bridging Science and Machine Learning for Light Curve Analysis). Rotation periods of a few minutes there play the same role as oscillation periods, and the task becomes classification under a total dwell budget.
Baselines
The naive reference is the survey schedule: split $T$ equally across targets, give each one contiguous window, then apply a fixed 4σ threshold. This is what TESS-style campaigns effectively do. Any adaptive method must beat it on utility $U$ at equal $T$, with the same realized gaps and slews.
The strongest hand-built competitor is greedy expected-information-gain-per-unit-time scheduling with remaining-time awareness. At each step it picks the target and dwell increment that most reduces posterior entropy over $Q_i$, using the same red-noise-calibrated evidence model. We consider it the strongest because, when information accrues smoothly, myopic information-gain scheduling is close to optimal. No reported result was found on Lacuna for adaptive campaign scheduling on TESS-like light curves, so this baseline is defined here rather than quoted.
The strongest learned competitor is OCMDP adapted to step-level observe-or-not decisions, trained with its published Lagrangian-style cost penalty (OCMDP: Observation-Constrained Markov Decision Process). In its own healthcare domain, OCMDP cut observation costs by 50% against always-observing and improved expected reward by 71% over PPO on Diagnostic Chain. Those figures are not transferable here. We will give it, and a recurrent approximate-information-state variant following Patil et al., the same simulator interactions and hyperparameter search budget as the planner's model fitting. We will report the rate at which each method exceeds the budget.
The key ablation is the identical tree-search planner with its history state stripped to remaining time and current significance. In this ablation the per-target contiguous-span and gap counts are replaced by total time spent on each target. If this matches the full planner, our central claim is wrong.
On RoBo6, Astroconformer's reported 72.5% accuracy on full light curves (LCDC: Bridging Science and Machine Learning for Light Curve Analysis) serves as the full-observation ceiling. Beating a baseline means a higher mean $U$, or higher accuracy at matched budget, with non-overlapping 95% bootstrap intervals over at least 1,000 simulated campaigns. It also means the false-claim rate does not rise.
Why not just…?
Why not just use greedy information-gain scheduling? This is the real threat. If evidence accumulated smoothly with time, greedy scheduling would be near-optimal and our planner unnecessary. Our argument rests on a step-function effect: a dwell shorter than one cycle yields roughly zero expected gain, so a myopic scheduler should keep switching to targets with cheap marginal gains. The experiments settle this by sweeping the ratio of oscillation period to minimum dwell length. At small ratios we predict the greedy scheduler ties the planner, and we will report that tie as a boundary of the claim. At ratios above about 2 we predict a widening gap.
Why not just fix a contiguous window of k cycles per target? A static rule that is well tuned on a homogeneous population could be hard to beat. It cannot drop a target early once strong red noise has made a detection hopeless, and it cannot extend a target whose oscillation is just emerging. We will tune $k$ by oracle on the test distribution, which favors the rule. We will then compare on populations that mix spectral slopes across the 1.7–3.2 range. The prediction is that adaptivity pays only under that heterogeneity.
Why not just run a sequential probability ratio test per target? Sequential testing is the classical solution to deciding whether to keep watching. However, it assumes independent increments of evidence and a single target. Red noise makes increments strongly correlated, and the shared budget couples targets. We include a Monte Carlo–calibrated sequential test with pooled budget as an additional baseline. If it matches the planner, the contribution reduces to the calibration lookup, and we will say so.
Why not just plan over a full belief state? A full belief is sufficient in principle, but a posterior over spectral slope, oscillation period, amplitude, and on/off times for each target is expensive to plan over. We run a particle-belief planner on small campaigns of 3–5 targets as an upper reference. The compact state must come within 5% of it.
Decisive Checks
Four checks decide the outcome. First, the ablation without span information must lose clearly, and only in the regime of long periods relative to minimum dwell. Second, the planner must keep its false-claim rate at or below the survey schedule's while recovering more true oscillations. False claims are the failure that short windows induce, so this is the check that matters most scientifically.
Third, in a replay test on archived TESS sectors 11 and 38 of PKS 1440-389, the planner reveals segments of the real light curves. It should reach the reported >4σ conclusions of He et al. while spending fewer days. It should also decline to claim oscillations on sources where Pininti et al. found none. Fourth, on RoBo6, accuracy at half the total dwell should approach the 72.5% full-curve reference more closely than fixed-dwell allocation does.
References
- Exploring short-term optical variability of blazars using TESS
- Optical Quasi-Periodic Oscillation of Blazar PKS 1440-389 in the TESS Light Curve
- Anytime-Constrained Reinforcement Learning
- Solving General-Utility Markov Decision Processes in the Single-Trial Regime with Online Planning
- OCMDP: Observation-Constrained Markov Decision Process
- Learning When to Observe: A Frugal Reinforcement Learning Framework for a High-Cost World
- Active Measure Reinforcement Learning for Observation Cost Minimization
- On learning history-based policies for controlling Markov decision processes
- LCDC: Bridging Science and Machine Learning for Light Curve Analysis