When a connected car moves across a city, the app it relies on (a navigation service, an augmented-reality overlay, a "digital twin" copy of the vehicle) runs on a nearby edge server. To keep delay low, that app's live state can be moved, or migrated, to a server closer to the car. Every migration has a cost: memory and context have to cross the network, and the service pauses while that happens. Learned migration policies already balance this cost against latency well, but all of them assume they know exactly where the car is. In practice the position comes from GPS, which jumps, drifts, and drops out in dense streets. A policy trained on clean positions will read a jittering fix near a coverage boundary as real movement and migrate the service back and forth. Each of those moves costs a full state transfer.
This note proposes training migration policies to handle noisy location input through a curriculum, meaning an ordered training schedule that starts easy and gets harder. Difficulty is set jointly by two quantities: how noisy the position estimate is, and how expensive a wrong migration would be, which grows with the size of the service's state. The policy also receives the location filter's own uncertainty as input. The claim is that a migration policy should be cautious only when uncertainty and migration cost are both high, and eager otherwise. We expect a policy trained from the start on uniformly noisy data to settle into a blanket "rarely migrate" behavior that it never unlearns. Every element of this claim can be tested against cheap classical alternatives, and the note is set up so that those alternatives can win.
Core Claim
The harm that location noise does to migration decisions is asymmetric and depends on state size. Waiting too long costs a little latency at each step. Migrating wrongly costs a transfer and downtime in proportion to the state footprint, often followed by a second transfer to undo it. Write the per-step cost as
$$c_t = L(\delta_t) + \mathbb{1}[a_t = \text{migrate}]\,(\alpha S_t + D),$$
where $\delta_t$ is the true network distance between user and serving node, $L$ is the resulting latency, $a_t$ is the action, $S_t$ is the current state size, $\alpha$ is transfer cost per byte, and $D$ is a fixed downtime penalty.
The policy sees only a noisy estimate $\hat\delta_t$, with filter uncertainty $\sigma_t$. Under this cost, the best "wait margin" before migrating should grow roughly with $\sigma_t$ times $S_t$. A fixed hysteresis threshold cannot represent that dependence, and a clean-trained policy never learns it.
Our hypothesis has two parts. A curriculum over the difficulty index $d = \sigma \cdot S$, with a rehearsal buffer of clean episodes, should learn this margin. Uniform noise injection, by contrast, should either fail to converge or collapse toward never migrating. The research file names this collapse as the "domain randomization trap."
Relation to Prior Work
The migration line motivates the problem and supplies the base learner. Dynamic Service Migration in Mobile Edge-Clouds reduces 2D movement to a 1D distance Markov decision process (a model where the next state depends only on the current state and action) and solves it about 1,000 times faster. It reports up to 44% lower cost than always-migrate or never-migrate. It assumes the hop distance is known exactly.
Mobility-aware Seamless Service Migration and Resource Allocation in Multi-edge IoV Systems, called SR-CL in that paper, splits the problem in two. An actor-critic reinforcement-learning agent decides when and where to migrate, and an analytical KKT step (the standard optimality conditions for constrained convex problems) allocates CPU and bandwidth. It runs on Simu5G with Rome taxi traces. Efficient Twin Migration in Vehicular Metaverses splits a multi-agent policy between vehicle and roadside unit and trains it on trajectories generated from T-Drive that are aligned to the road map.
All three treat positions as ground truth and train without any schedule. We keep their decision structure and change what the policy observes and how it is trained.
Multi-armed Bandits for Self-distributing Stateful Services constrains our design. Its UCB1 bandit (a standard explore-versus-exploit rule) fails to converge when state size drifts. That is why we treat $S_t$ as a curriculum axis rather than a fixed constant.
The curriculum literature supplies the method. Self-Paced Learning for Latent Variable Models shows that admitting easy samples first avoids bad local minima. It got trapped once in 100 DNA-motif runs, against 18 for its baseline. It also warns that the pacing schedule is a sensitive hyperparameter, which we address by pacing automatically on a performance plateau.
Benchmarking Massively Parallelized Multi-Task RL finds that sparse-reward tasks fail without a curriculum. Migration rewards are not sparse, though, so we do not lean on this result. Our argument is about collapse into a conservative policy, not about exploration.
DemoStart and Adaptive Curriculum Generation from Demonstrations widen the distribution of starting states for robot sim-to-real transfer. We widen the observation-noise and cost distribution instead, and the actuator is a network decision rather than a motor.
The robustness literature motivates feeding the policy an abstracted, uncertainty-aware input instead of raw signals. Deep Drone Acrobatics reaches 100% versus 0% success by giving the policy feature tracks instead of pixels. Uncertainty Quantification with Statistical Guarantees shows that uncertainty-aware control survives out-of-distribution weather that breaks deterministic models. We borrow the cheap version of this idea: the filter's covariance is passed as an input, so no expensive Bayesian inference is needed on edge hardware.
Research Program
We build three things.
The first is a noisy-observation migration benchmark. Simulated ground truth comes from the Rome traces in Simu5G, the SR-CL setting. On top of it we add structured GPS corruption: bursty multipath error, dropouts, and slow drift, rather than independent Gaussian noise. We also build a test set from real noise. We propose using raw T-Drive logs as observations and their road-matched versions as approximate truth. Policies are trained only on the synthetic noise and evaluated on that real pairing, which gives a genuine within-data sim-to-real gap.
The second is the policy itself. It is SR-CL's actor-critic with the KKT allocation unchanged. Its input is extended to include the filtered distance estimate, the filter covariance $\sigma_t$, and the current state size $S_t$.
The third is the curriculum. Training starts at $d \approx 0$, with clean positions and small states. Difficulty advances whenever the rolling validation cost plateaus. Throughout, 20% of episodes are replayed from earlier difficulty levels, to guard against forgetting.
The analysis then probes the learned policy directly. We sweep $\sigma$ and $S$ and record the effective distance at which the policy migrates. This tests whether the learned margin actually scales with their product, as the cost model predicts.
Baselines
The naive reference points are always-migrate and never-migrate. Dynamic Service Migration in Mobile Edge-Clouds reports that its MDP policy costs up to 44% less than these heuristics on clean San Francisco traces. Any method here must beat them on noisy input by a clear margin, or it has learned nothing.
The strongest learned competitor is SR-CL from Mobility-aware Seamless Service Migration. It reports a 3.63% to 36.15% reduction in total system delay against its benchmarks across 25 nodes, with decisions taking under 1 ms for 220 vehicles, on clean Rome traces. We found no reported result on Lacuna for SR-CL, or for any migration policy, under noisy location input.
We will run SR-CL in two forms with the same network, data, environment steps, and tuning budget as our method. In one form it is trained on clean data and tested on noisy data. In the other it is trained with uniform domain randomization at the final noise and state-size distribution. Beating SR-CL means two things together. On noisy and raw-T-Drive test sets, our total system delay including migration cost must be lower by more than the variance across seeds. On clean input, our delay must be within a few percent of clean-trained SR-CL.
The strongest non-learned competitor runs an extended Kalman filter on the client. The filtered distance feeds the 1D distance MDP, which is solved on the belief over distance, and a hysteresis margin is grid-tuned separately for each state-size bucket. This baseline gets the same tuning budget as our curriculum's hyperparameters.
Two ablations isolate the key ingredients. One trains uniformly but with the covariance input, which isolates the curriculum. The other runs the curriculum without the covariance input, which isolates the uncertainty signal. A third variant schedules noise alone and leaves state size out, which tests whether the joint difficulty index matters. Finally, a policy given true positions provides the upper bound.
Why not just…?
Why not just inject Gaussian noise uniformly during training (domain randomization)? This is the most direct threat. Migration rewards are dense, so the exploration argument from Benchmarking Massively Parallelized Multi-Task RL does not transfer. Our case depends entirely on whether uniform noise produces a conservative, low-migration local optimum. The "uniform + covariance" ablation settles this. If it matches the curriculum on cost and on clean-input eagerness, we drop the curriculum. The contribution would then shrink to the benchmark and the input design, and we would report it that way.
Why not just Kalman-filter the GPS and keep the existing policy? Filtering reduces noise but does not tell the decision rule how much to distrust the estimate, and it does nothing about the cost of a wrong move growing with state size. The relevant evidence in the file, Deep Local Trajectory Replanning and Control beating ROS Nav Stack by 84% to 66% on unseen maps, comes from robot navigation, so it only suggests the gap. The filter-plus-belief-MDP-plus-tuned-hysteresis baseline is designed to test this directly. If per-bucket tuned hysteresis matches our policy, the honest conclusion is that a two-parameter rule is enough, and we will say so.
Why not just use the 1D distance MDP with a hysteresis threshold? On clean input it is near-optimal and very cheap. Its weaknesses here are that it assumes the distance is known and that state size is fixed. Solving it on the belief over distance addresses the first weakness, which is why that version is included as a baseline rather than dismissed. The state-size sweep tests the second. It shows whether a single solved MDP degrades as $S_t$ drifts, as the bandit in Multi-armed Bandits for Self-distributing Stateful Services did.
Proposed Experiments
All methods are evaluated on Rome/Simu5G under synthetic structured noise at three severity levels, and on raw versus road-matched T-Drive. State size is either fixed or drifting.
Metrics are total system delay, migration count, bytes transferred, cumulative downtime, and flicker rate. Flicker rate is the fraction of migrations reversed within a short window, and it measures the specific failure that noise causes. We also report the clean-to-noisy degradation gap.
Three checks are decisive. First, plot migration rate against training step for uniform randomization and for the curriculum. If uniform randomization does not collapse toward never-migrate, the core mechanism is wrong. Second, on raw T-Drive, compare our policy with the tuned filter-plus-hysteresis rule. A meaningful win there is the main evidence that learning is worth deploying. Third, probe the learned migration margin as a function of $\sigma \times S$. A margin that grows with that product confirms that the policy learned the cost-coupled caution we predicted. A margin that ignores $S$ means the joint curriculum added nothing over a noise-only schedule.
References
- Dynamic service migration in mobile edge-clouds
- Mobility-Aware Seamless Service Migration and Resource Allocation in Multi-Edge IoV Systems
- Efficient Twin Migration in Vehicular Metaverses: Multi-Agent Split Deep Reinforcement Learning With Spatio-Temporal Trajectory Generation
- Multi-armed Bandits for Self-distributing Stateful Services across Networking Infrastructures
- Self-Paced Learning for Latent Variable Models
- Benchmarking Massively Parallelized Multi-Task Reinforcement Learning for Robotics Tasks
- DemoStart: Demonstration-led auto-curriculum applied to sim-to-real with multi-fingered robots
- Adaptive Curriculum Generation from Demonstrations for Sim-to-Real Visuomotor Control
- Deep Drone Acrobatics (Extended Abstract)
- Uncertainty Quantification with Statistical Guarantees in End-to-End Autonomous Driving Control
- Deep Local Trajectory Replanning and Control for Robot Navigation