Introduction
Persistent monitoring involves a fleet of mobile agents tasked with tracking the states of multiple targets over an infinite time horizon. In scenarios such as traffic monitoring at city intersections or environmental sensing in rivers, the number of targets often exceeds the number of available agents. Consequently, agents must move between targets to "refresh" their state estimates, as uncertainty—modeled as estimation error—grows when a target is left unobserved. While much existing research focuses on short-term transient planning, this work addresses the need for long-term autonomy by seeking optimal steady-state periodic trajectories.
Problem Formulation
The system consists of targets with internal states evolving according to stochastic linear dynamics. To estimate these states, the agents employ a Kalman-Bucy filter. The primary objective is to minimize the mean quadratic estimation error across all targets. This error is tracked via a differential Riccati equation, which describes how uncertainty increases due to process noise and decreases when an agent is within sensing range of a target.
Methodology
The authors propose a framework for optimizing periodic schedules using several key components:
Estimation and Sensing Model
The quality of information gathered by an agent depends on its proximity to a target. As shown in Figure 1, the Signal-to-Noise Ratio (SNR) is maximized when the agent is directly positioned at the target location ($s_j - x_i = 0$) and decreases linearly to zero as the distance reaches the sensing range $r_j$. This sensing model directly influences the estimation error dynamics.
Trajectory Parameterization
For targets arranged on a 1D line, the authors prove that the optimal movement policy follows a "move or dwell" structure. Agents should either travel at their maximum speed between targets or dwell (remain stationary) at a target location to maximize sensing effectiveness. This simplifies the optimization problem from searching a continuous space of paths to tuning a finite set of parameters: the total period ($T$), initial positions, dwelling times, and travel times.
Infinitesimal Perturbation Analysis (IPA)
To optimize these parameters, the researchers utilize Infinitesimal Perturbation Analysis (IPA). IPA provides an efficient way to calculate the gradient of the steady-state cost function. Because IPA is event-driven—updating only when an agent starts or stops moving—it is highly scalable and avoids the computational burden of traditional long-term path planning. These gradients are then used in a standard gradient-descent algorithm to iteratively refine the trajectory.
Key Insights
- Steady-State Optimization: By focusing on the limit cycle rather than the initial "warm-up" phase, the complexity of the long-term plan is significantly reduced.
- Global Attractivity: Provided every target is visited at least once, the estimation error converges to a unique periodic steady state regardless of initial conditions.
- Automatic Workload Partitioning: In multi-agent scenarios, the optimization naturally leads to a division of labor. For instance, in a five-target, two-agent simulation, the agents self-organize to cover distinct subsets of targets, minimizing redundant travel.
Results
Simulation results across various scenarios (e.g., one agent for two targets, two agents for five targets) demonstrate the effectiveness of the IPA-driven approach. In a two-target case, the algorithm converged to a symmetric policy where the agent spent equal time dwelling on each target and moving between them, drastically reducing the error compared to initial heuristic guesses. The method proved scalable, handling infinite horizons with minimal parameterization.
Conclusion and Future Work
This work provides a robust framework for long-term persistent monitoring. Future research will focus on extending the "move or dwell" proof to 2D and 3D environments, addressing the challenges of local optima in gradient descent, and incorporating more complex target dynamics or agent constraints such as acceleration limits.
Summary updated