Calibrating the Replacement Decision: Threshold-Local Calibration for Thermally Accelerated Tool Wear

When a cutting tool or protective coating nears the end of its life, a maintenance system has to answer one question: what is the probability that wear will cross the replacement limit within the next few operations? If the system says 10% and the true frequency is 40%, parts break in service. Deep models for tool wear currently output point estimates, and generic uncertainty methods are calibrated on average over the whole tool life. Most of that life is the long, quiet middle stage, where calibration is easy and the decision does not matter. We propose to train wear forecasters with a differentiable penalty that enforces calibration only where the decision lives. That region is the late stage, where frictional heating softens the tool and oxide layers form and scrape off, wear accelerates, and the relationship between sensor signals and wear changes. The penalty targets the forecast probability of crossing the limit and is localized by a kernel to forecasts close to that limit.

The claim is that a forecaster can look well calibrated on marginal metrics and still be dangerously overconfident at the replacement threshold. The claim also says two things about fixing this: post-hoc recalibration can close the gap only by widening its forecasts, and an in-training, threshold-localized penalty can close it while keeping forecasts sharp. The central empirical question is whether in-training localization beats a stage-conditional post-hoc correction. That question has a clean answer, and the proposal is designed to accept either outcome.

Why the gap is real, and what already exists

The degradation literature explains why the late stage is different. Coating studies show high temperature changes the physics of failure. In High Temperature Oxidation Resistance and Electrical Properties of CrN/Cr Films Prepared by RF Magnetron Technology, CrN films annealed at 800 °C decompose into Cr₂N and brittle oxides. In Effect of B Element Doping on High-Temperature Tribological Properties of WS2-Based Composite Coatings, the wear mechanism of WS₂ coatings shifts from adhesive wear to oxide-particle rolling and mixed abrasive/adhesive wear as temperature rises. These papers motivate the proposal: the mapping from load and sensor signals to wear is not stable across thermal regimes. A model fit mostly on steady-stage data should therefore be least reliable exactly where wear accelerates. We do not model these chemistries directly. They explain why the calibration failure should concentrate near end of life.

On the prognostics side, Tool Wear Prediction Based on Residual Connection and Temporal Networks introduces DResNet-1d. It is a residual 1D convolutional network with an attention-based recurrent encoder-decoder, forecasting flank wear on PHM2010 milling data, and it is evaluated only by MAE and RMSE. Intelligent Life Prediction of Thermal Barrier Coating for Aero Engine Blades classifies coating states from SEM images with uncalibrated softmax outputs. Uncertainty-aware Remaining Useful Life predictor is the closest probabilistic work. It uses deep Gaussian processes and sigma-point processes on turbofan data, optimizes variational likelihood bounds, and reports NLL. It does not measure or train for calibration of a threshold event. The Hawkes-process corrosion work, Modeling Discrete Coating Degradation Events via Hawkes Processes, forecasts failure times with interval widths but no calibration objective. All four motivate the gap, and none supplies a calibration mechanism.

The mechanism comes from the calibration literature. Calibration by Distribution Matching: Trainable Kernel Calibration Metrics shows that kernel-based calibration errors can be estimated without bias and differentiated, and that the kernel can be chosen to target decisions. We reuse that construction directly. Our proposal is therefore a specialization, not a new family of estimators. The contribution is the choice of calibration target (a horizon-specific exceedance event), the choice of localizing variable (distance to the replacement limit), and a controlled test of whether this matters on real degradation data. A Large-Scale Study of Probabilistic Calibration in Neural Network Regression constrains the design. Across 57 tabular datasets, it finds that post-hoc empirical recalibration and conformalized quantile regression reach near-perfect probabilistic calibration by inflating intervals, while an in-training kernel-density calibration penalty (PCE-KDE) preserves sharpness. That result is marginal and tabular; we test whether it transfers to the conditional, temporal, shifted setting. X-CAL: Explicit Calibration for Survival Analysis softens survival-calibration bins so they can be trained against. It is related because "crosses the limit within h steps" is a survival event, but X-CAL calibrates the whole time-to-event distribution marginally rather than near a decision threshold. Smooth Calibration Error: Uniform Convergence and Functional Gradient Analysis shows proper losses implicitly control smooth calibration error but degrade with overfitting on finite data. This is the theoretical reason a plain likelihood-trained model is not enough, and smooth calibration error also serves as an evaluation metric.

Core Claim

Let the forecaster output a predictive distribution for wear $h$ steps ahead, and let $p_i$ be the implied probability that wear exceeds the replacement limit $L$ at example $i$. Let $y_i \in \{0,1\}$ record whether it actually did, and let $z_i$ be the forecaster's predicted median distance to $L$. We add to the likelihood a threshold-local kernel calibration error:

$$\mathrm{LCE}^2 = \frac{1}{B(B-1)} \sum_{i \neq j} (y_i - p_i)(y_j - p_j)\, k_p(p_i, p_j)\, k_z(z_i, z_j)\, w(z_i)\, w(z_j),$$

where $B$ is the batch size, $k_p$ and $k_z$ are Gaussian kernels on the forecast probability and on distance-to-limit, and $w$ is a weight concentrated near $z=0$. This quantity is zero in expectation when the event frequency matches $p$ among forecasts that are near the limit. It is differentiable in $p$ because the hard labels $y$ carry no gradient. It is the same U-statistic form as the kernel calibration metrics of Marx et al. The training loss is NLL plus $\lambda \cdot \mathrm{LCE}^2$, with the penalty summed over a small set of horizons and thresholds. Setting $k_z$ and $w$ to constants recovers an unlocalized kernel calibration of the exceedance probability, which gives us a clean ablation.

Research Program

The backbone is a DResNet-1d-style encoder over the seven PHM2010 channels: three cutting-force axes, three vibration axes, and acoustic emission. It feeds a mixture-density head over future wear at horizons of 5, 10 and 15 cuts, matching the horizons in the DResNet-1d paper. The PHM2010 data provide wear labels for only three cutters (C1, C4, C6), so all evaluation is leave-one-cutter-out. Holding out a cutter makes the test tool a different physical run with its own wear curve, which gives a real, if small, form of the drift the coating papers describe. Thresholds $L$ are set at several fractions of each training cutter's final wear, so the event is defined without reference to the held-out tool.

To test generality, we repeat the study on N-CMAPSS turbofan degradation. There the event is "remaining useful life below a fixed cycle count," the backbone is replaced by the feed-forward and probabilistic models of the uncertainty-aware RUL paper, and Unit 14 serves as the out-of-distribution test that paper already uses.

The analysis has three parts. First, we measure whether the gap exists. We compare calibration of the exceedance probability overall against calibration in the near-threshold, accelerated-wear region, for likelihood-trained models before any correction. Second, we compare how to fix it: in-training versus post-hoc, and localized versus global. Third, we translate calibration into decisions. We simulate a replace-when-$p$-exceeds-$c$ policy under cost ratios between in-service failure and early replacement, and report expected cost per tool.

Baselines

The naive reference is the same backbone and mixture head trained on NLL alone, with exceedance probabilities read off its predictive distribution. This is what practitioners would do after adding a probabilistic head to DResNet-1d. Every method shares this backbone, data split, training epochs, and a matched hyperparameter search budget (the same number of trials for $\lambda$, kernel bandwidths, or post-hoc settings). All post-hoc methods get the same held-out calibration cutter.

The strongest point-prediction reference is DResNet-1d itself. On cutter C1 it reports MAE 2.09 and RMSE 2.58 with temporal smoothing, and MAE 3.76 and RMSE 4.99 at 15 steps ahead (Tool Wear Prediction Based on Residual Connection and Temporal Networks). No reported calibration or interval result was found on Lacuna for PHM2010. We therefore require only that our model stay within noise of these point errors, not beat them.

On N-CMAPSS, the strongest reported probabilistic model is the deep sigma point process. It reports RMSE 7.38 and NLL 3.10, against MC Dropout's RMSE 7.31 and NLL 4.26 (Uncertainty-aware Remaining Useful Life predictor). We rerun it and compute threshold-local calibration of its implied exceedance probabilities. Beating it means lower near-threshold calibration error at NLL no worse than 3.10.

The post-hoc competitors are four. Isotonic recalibration of the exceedance probability is the classification analogue of temperature scaling. Conformalized quantile regression is run on the wear forecast. A stage-conditional version of each is fit separately on calibration points near the threshold. Stage-conditional post-hoc correction is the strongest competitor on this question, because it targets the same region we do without touching training.

The in-training competitors are two. PCE-KDE, following Dheur and Ben Taieb, calibrates the full predictive distribution marginally. The unlocalized kernel penalty, following Marx et al., calibrates the exceedance probability globally. The second is also our key ablation: it differs from the proposal only in dropping $k_z$ and $w$, so it isolates the effect of localization.

Beating a baseline means lower threshold-local calibration error on held-out cutters, with confidence intervals from a block bootstrap over time within each cutter. It also means a Brier resolution term (sharpness) at least as good, and lower simulated maintenance cost over a range of cost ratios. Winning on calibration alone while losing sharpness does not count.

Why not just…?

Why not just apply temperature scaling or isotonic regression to the exceedance probability? A single global correction fits whatever miscalibration dominates the calibration set, and here that is the steady stage. The expectation-consistency work, Expectation Consistency Loss: Rethink Confidence Calibration under Covariate Shift, shows source-fitted corrections break under shift: for example, it reports ECE of 61.9% on SVHN shift that its in-training method reduces to 21.5%. Held-out cutters are a shift. The global post-hoc baselines test this directly. If they already match the localized penalty near the threshold, the premise of this proposal is wrong, and we would report that.

Why not just use conformal prediction or recalibration conditioned on the late stage? This is the real threat. Stage-conditional conformal prediction gives near-threshold coverage by construction, needs no retraining, and makes a distribution-free guarantee we cannot. Its weakness is data: with three labeled cutters, the near-threshold calibration set is a few dozen correlated points. The Dheur and Ben Taieb evidence suggests it will buy coverage by widening intervals. Our comparison measures exactly this: coverage and calibration at matched sharpness, and decision cost. If stage-conditional conformal ties the localized penalty on cost, the practical recommendation becomes "use conditional conformal." The in-training penalty would then be of interest only if it lowers the width conformal needs when the two are stacked, which we also test.

Why not just train on likelihood and rely on proper scoring rules to calibrate? Futami and Nitanda show proper losses control smooth calibration error only up to a complexity term that grows with training. On a few hundred cuts per tool, overfitting is the expected regime. The naive baseline measures whether this matters. The first decisive check below is built on it.

Decisive Checks

The project hinges on three results in order. First, the likelihood-only model must show clearly worse exceedance calibration near the threshold than overall on held-out cutters, beyond bootstrap noise. If it does not, the gap is not there and the project stops at that finding. Second, the localized penalty must beat its unlocalized ablation near the threshold. If it does not, localization adds nothing over Marx et al. Third, the localized penalty must match or beat stage-conditional conformal prediction on simulated maintenance cost at equal or better sharpness, both on PHM2010 and on N-CMAPSS Unit 14. Passing all three would support the claim. Failing the third would still yield a useful negative result: for thermally accelerated wear, conditioning the post-hoc correction is enough.