When a laser, torch, or other concentrated heat source is moved across a part, heat piles up wherever the path turns back on itself: at scan turnarounds, sharp corners, and the centers of shrinking contours. The usual remedy is to speed up at these "heat traps" so that less energy lands on already-hot material. When the heat source is carried by a gantry or robot arm instead of an almost weightless mirror, speeding up is limited by how fast the machine can accelerate. Commanding a speed change the machine cannot follow makes it overshoot the path. Slowing down to stay gentle only makes the heat pile up more. We propose to treat this as a constrained learning problem with two safety limits: the melt pool must stay within a depth band, and the machine must stay in the regime where it tracks the commanded path. The central idea is that satisfying both limits requires anticipation. The controller must start accelerating before it reaches a heat trap, and it can only do that if it forecasts how the temperature field will evolve along the path ahead.
Concretely, we will train a spatiotemporal forecaster of the thermal field. It is a neural network that predicts how temperature spreads over the part geometry several steps into the future. We will use it as a safety filter during reinforcement learning. The filter looks at the policy's proposed speed profile, predicts melt-pool depth along the upcoming path, and replaces the action with the nearest acceptable one if a violation is forecast. The policy learns to maximize build speed and depth accuracy inside this filter. The claim that would be new is quantitative. As the machine's acceleration limit tightens, the lead time the controller needs grows, and methods that react only to the current state should fail sharply at a predictable point. A forecast-based filter should hold up until the required lead time exceeds its forecast horizon.
Core Claim
The closest prior work, Thermal Control of Laser Powder Bed Fusion Using Deep Reinforcement Learning, trains unconstrained proximal policy optimization (a standard policy-gradient method, PPO) to modulate laser velocity. Its state is nine 2D temperature slices at the current and two previous timesteps. It encodes the 55 µm melt-pool depth target as a shaped reward penalty, and it simulates heat cheaply by superposing stored semi-analytical Eagar–Tsai line solutions (the RUSLS method). The policy learned to speed up toward the center of concentric triangles. The paper leaves four things out. It has no formal safety constraint. It says nothing about how many violations occur during training. Its velocity is kinematic, meaning the heat source goes wherever and as fast as commanded. And its state window looks only backward in time.
Our proposal differs in each of these respects. We give the machine carrying the heat source a finite acceleration limit $a_{\max}$. Beyond that limit, the quasi-static idealization breaks: that idealization, common in manipulation planning, assumes the mechanism follows its commanded motion with no lag, overshoot, or momentum. We treat both depth accuracy and path tracking as constraints in a constrained Markov decision process, which is a reinforcement learning problem where expected cumulative safety cost must stay under a budget. We enforce the thermal constraint during learning, using a forecast of the field rather than a snapshot of its past.
The mechanism rests on a simple kinematic fact. If a heat trap requires raising speed by $\Delta v$, and the machine can accelerate at most $a_{\max}$, the controller must begin accelerating at least
$$\tau_{\text{lead}} \geq \frac{\Delta v}{a_{\max}}$$
seconds before reaching the trap. The bound follows directly from constant-acceleration kinematics. A controller that sees only the current and recent field cannot act that early unless it has implicitly learned the geometry-to-heat mapping from many violations. A controller that forecasts $H$ steps ahead can act early as long as $H\,\Delta t \geq \tau_{\text{lead}}$, where $\Delta t$ is the control step. This gives a falsifiable prediction: the $a_{\max}$ below which each method starts to fail.
Research Program
The environment builds on the RUSLS-style Eagar–Tsai superposition simulator from the LPBF deep RL paper. That simulator is not listed in the file as a released resource, so we will reimplement it from the paper's description and check it against its reported constant-velocity behavior.
Between the policy's commanded speed and the actual heat-source position, we insert a second-order tracking model of a gantry or arm axis with an acceleration limit. This models the setting of robot-carried directed energy deposition, the process studied by VIMPS. There, overshoot at corners displaces the heat source from the intended path. The mechanical model is deliberately simple. We are not modeling contact-rich manipulation. We borrow one lesson from that literature: planners that assume quasi-static motion miss momentum-driven residual motion, as shown for sliding in Learning to Slide Unknown Objects with Differentiable Physics Simulations. Unmodeled inertial effects under heavy payloads cause oscillation (Osumi et al.). Our tracking-error cost measures directly how far the policy has pushed the machine outside the regime where commanded and actual paths agree.
The constrained problem has one reward and two costs. The reward pays for build speed and penalizes squared depth error. The first cost fires when melt-pool depth leaves a band around the target: too shallow risks lack of fusion, and too deep risks keyholing. The second cost fires when tracking error exceeds a tolerance. We use a first-order constrained policy optimizer, following FOCOPS (First Order Constrained Optimization in Policy Space). FOCOPS computes a nonparametric improved policy that trades reward against cost through a Lagrange multiplier, then projects it back onto the network. It avoids the second-order computations of constrained policy optimization (CPO). A Lagrangian formulation matters here because Quanyi Li et al. show that fixed reward penalties either allow violations or produce policies that do nothing useful. FOCOPS, however, guarantees the constraint only near convergence. That is the gap our filter fills.
The forecaster is trained offline on simulator rollouts under randomized speed profiles and part geometries. It takes the current temperature field on a graph whose nodes are points of the part and whose edges follow material connectivity. It also takes a candidate speed profile for the next $H$ steps. It outputs predicted melt-pool depth along that stretch of path. We use the gated temporal convolution plus graph convolution structure of STGCN, with one change: the heat-source position is a moving input node rather than a fixed sensor. A small ensemble supplies an uncertainty margin.
At each step, the filter checks the policy's proposed action together with a default continuation. If the upper confidence bound on forecast depth exits the band, the filter searches a short list of feasible acceleration profiles for the one closest to the proposal that stays safe. The idea is related to shielding, Safe Reinforcement Learning via Shielding, where a verified monitor overrides unsafe actions in a discrete abstraction. Our filter trades that formal guarantee for applicability to a continuous field. It is closer in spirit to learned-model shields such as Adaptive Probabilistic Shielding. We will state plainly that the guarantee is only as good as the forecaster's calibration. The forecaster is refit online on observed fields, and we will test this under simulator mismatch.
Baselines
The naive reference is constant commanded speed. On this simulator family, the cited paper reports that constant velocity "showed melt pool depth spikes of up to 20 µm" on horizontal cross-hatching. On concentric triangles, "melt pool depth increased by 40 µm (a 72% error)" (Thermal Control of LPBF Using DRL). Our reimplementation must reproduce roughly these numbers before any other comparison is trusted.
The strongest existing method on this exact task is the reward-shaped PPO of that same paper. We will rerun it with its nine-slice, three-timestep state and its reward, now behind our acceleration-limited tracking model. The paper describes its result only qualitatively ("a tight distribution around the 55 µm target"). No numerical depth-error result is reported in the file, so we will report our own rerun. This baseline is the one to beat, because it already contains the "speed up at heat traps" behavior. It differs from our method only in having no constraint, no filter, and no forward-looking model.
The strongest constrained-learning competitor is FOCOPS, or PPO-Lagrangian from the SafePO library in Safety Gymnasium, using the same two costs but no filter. On Humanoid-Circle, FOCOPS is reported at a reward of 1106.1 against CPO's 671.0 within a cost limit of 50. On Safety Gymnasium velocity tasks, PPO-Lag is reported at a 98% reduction in violations with a 45% reward drop. These are rigid-body tasks, and no reported result was found on Lacuna for constrained RL on a thermal melt-pool task. We will reproduce the PPO-Lag velocity-task figure first, as a check that our constrained learner is implemented correctly.
The strongest non-learning competitor is model predictive control (MPC), which at each step optimizes a short future speed profile. It will use the analytic Eagar–Tsai model itself as the predictor, under the same acceleration limit and the same per-step compute budget as our filter.
The ablations isolate the key ingredient. The first uses our filter with a one-step (non-anticipatory) forecast. The second replaces the graph forecaster with a grid convolutional network plus a recurrent network, trained on the same data. The third removes the tracking constraint.
All learning methods share the same simulator steps, network size, and hyperparameter search budget. The search uses twenty configurations per method, selected on held-out geometries. Beating the field means two things together. The first is fewer cumulative depth-band violations during training than FOCOPS with no filter, by at least an order of magnitude. The second is final depth error and tracking-violation rates at least as good as reward-shaped PPO and MPC across the acceleration sweep described below.
Why not just…?
Why not just run MPC on the physics model and skip learning? This is the most serious threat. In the matched simulator it may match our method, since the Eagar–Tsai superposition is fast and exact by construction there. Our argument for a learned forecaster is not accuracy in simulation. It is that the forecaster can be refit from observed temperature data when the analytic model is wrong. The cited paper notes that its model ignores Marangoni flow, spatter, and phase change. We will settle this with a mismatch test. The "true" environment uses perturbed absorptivity and diffusivity plus an added depth-dependent correction. MPC keeps the nominal model, and our forecaster is refit online. If MPC with the nominal model still matches us under mismatch, we will report that the learned component is unnecessary here and reduce the contribution to the constrained formulation plus the lead-time analysis.
Why not just cap speed low so the machine always stays in the quasi-static regime? Low speed keeps tracking error small but concentrates heat. The concentric-triangle result in the LPBF deep RL paper shows that holding speed fixed drives a 72% depth error at the center. Our sweep will show this directly as a speed-capped constant-velocity arm that satisfies the tracking cost while violating the thermal band.
Why not just modulate laser power, which has no inertia? This could dissolve the tension entirely, and the cited work only controls velocity. We will add power as a second action within a realistic range. The floor of that range is set by the depth band's shallow limit, since cutting power too far causes lack of fusion. We will then measure how much of the violation reduction power alone provides. If power alone keeps depth in band across the sweep, the mechanical-limit story matters only for processes with narrow power windows, and we will say so.
Why not just use a grid CNN with an LSTM instead of a graph model? ST-ResNet argues that decoupled convolution-then-recurrence models capture joint space-time structure poorly, and STGCN adds that recurrence trains slowly. On simple rectangular scans, however, a grid model may do just as well. The forecaster ablation decides this. We expect the graph to matter only on non-convex parts, where grid neighbors are not material neighbors, and we treat that as secondary to the forecast horizon.
Proposed Experiments
We evaluate on the two scan patterns from the cited LPBF paper, horizontal cross-hatching and concentric triangles. We add a set of held-out non-convex geometries (L-shapes, thin ribs, annuli) to test transfer. Metrics are the distribution of depth error around 55 µm, the fraction of path length outside the depth band, the cumulative count of band violations over training, the tracking-error violation rate, and total build time.
The decisive experiment is the acceleration sweep. We vary $a_{\max}$ across a range from effectively unlimited (mirror-like) down to strongly limited. For each method, we plot final and training-time violation rates against the lead time $\Delta v / a_{\max}$ implied by each geometry's heat traps. The hypothesis is confirmed if the two myopic methods, reward-shaped PPO and FOCOPS without a filter, degrade sharply once the required lead time exceeds a few control steps, while the forecast filter stays near zero until the lead time exceeds $H\,\Delta t$. Varying $H$ must shift that breakpoint accordingly. It is refuted if myopic methods hold up just as well at tight $a_{\max}$, which would mean their history window already encodes enough anticipation. It is also refuted if the filter's breakpoint does not move with $H$.
The second decisive check is the model-mismatch test against MPC. The third is the power-modulation test. Together they tell us whether the forecast filter, the constrained formulation, or neither is responsible for the gains.
References
- Thermal Control of Laser Powder Bed Fusion Using Deep Reinforcement Learning
- Monitoring Variability in Melt Pool Spatiotemporal Dynamics (VIMPS): Towards Proactive Humping Detection in Additive Manufacturing
- Learning to Slide Unknown Objects with Differentiable Physics Simulations
- Development of a Cooperative System Comprised of a Robot and a 7 D.O.F. Crane with Three Wires.
- First Order Constrained Optimization in Policy Space
- MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learning
- Spatio-Temporal Graph Convolutional Networks: A Deep Learning Framework for Traffic Forecasting
- Safe Reinforcement Learning via Shielding
- Adaptive Probabilistic Shielding by Learning MDPs for Safe Reinforcement Learning
- Safety Gymnasium: A Unified Safe Reinforcement Learning Benchmark
- Predicting citywide crowd flows using deep spatio-temporal residual networks