Physics-Shaped Causal Smoothing for Irrigation Sensors Under Real Noise

Drip irrigation in arid regions saves water only if the controller knows when the root zone has actually dried out. It usually learns this from a cheap soil-moisture probe and a fixed trigger level. Those probe readings are noisy in ways that change over time. Wetting pulses arrive as near-instant jumps, followed by slow drying that can last weeks. On top of that, soil temperature swings every day and shifts the probe's electrical reading on the same daily cycle. The usual fix is a moving average (replace each reading with the mean of a sliding window of recent readings) or its polynomial cousin, the Savitzky–Golay filter. Both treat rises and falls the same way. They blur the moment water arrives, and they let the daily temperature swing leak through, sometimes as spurious crossings of the trigger level.

We propose a causal smoother, meaning it uses only past readings, that replaces the polynomial inside a Savitzky–Golay window with the physical shape of a drying curve. It resets the window whenever a wetting jump is detected. It removes the daily temperature artifact by regressing on a co-located soil-temperature channel, instead of hoping a wide window averages it away. Our second claim is methodological, and we think it matters as much as the filter. Smoothers for this setting are almost always compared on simulated data with added Gaussian noise. We will test whether their ranking survives when the simulator instead replays noise residuals extracted from real field probes. If the ranking flips, simulation-only evaluation of irrigation sensing is misleading. That would be a finding in its own right, whichever smoother wins.

Core Claim

A symmetric fixed window is the wrong tool for soil moisture, and the reason is physical rather than a matter of tuning. A changepoint approach to modelling nonstationary soil moisture dynamics segments real NEON soil-moisture records into wetting jumps followed by exponential drydowns, each with its own asymptote and decay rate. It works offline, over the whole record. Annual Cropping Intensity Dynamics in China from 2001 to 2023 shows that a grid-searched Savitzky–Golay window of 5 to 7 composites preserves crop-phenology peaks. That smoothing is also offline, on 8-day satellite composites, far from a live irrigation valve.

Our claim is that fitting, inside a trailing window that restarts at each detected wetting event, the model

$$y(t) = \alpha + (\theta_c - \alpha)\,e^{-\gamma (t - t_c)} + \beta\,\Delta T(t) + \varepsilon(t)$$

yields three measurable improvements over moving-average and Savitzky–Golay filters matched for noise reduction. The symbols in this model are:

  • $y(t)$ is the probe reading at time $t$.
  • $t_c$ and $\theta_c$ are the time and moisture level of the last detected wetting event.
  • $\alpha$ is the level the soil dries toward, and $\gamma$ is the drying rate.
  • $\Delta T(t)$ is the soil-temperature deviation from its trailing daily mean, and $\beta$ is a coupling coefficient fitted online.
  • $\varepsilon(t)$ is residual noise, whose variance we track per window.

The three improvements are that the smoother reports water arrival sooner, crosses the irrigation trigger less often by mistake, and gives more stable estimates of the drying rate. This is the "minimum modification principle" from Time Series Data Cleaning: A Survey: smoothing should not alter genuine transitions. Here it is made concrete for a signal whose genuine transitions are one-sided jumps.

Research Program

The work has three parts.

The smoother. We build the smoother as a causal online procedure. A lightweight jump detector compares each new reading against the current window's prediction, scaled by the tracked residual variance. When a reading exceeds the prediction by a margin, the detector declares a wetting event and restarts the window. Between events, a small nonlinear least-squares update refits $\alpha$, $\gamma$ and $\beta$. The design borrows an idea from speech enhancement. Interactive Speech and Noise Modeling for Speech Enhancement models non-stationary noise as a structured signal in its own branch and loses about 0.5 dB SDR without that interaction. We keep the lesson and not the network: the daily temperature artifact gets its own explicit term, because it is structured. The whole procedure has to run on the Raspberry Pi-class hardware used in DESIGNING AN INTELLIGENT IRRIGATION SYSTEM BY USING BACKPROPAGATION NEURAL NETWORK TO PREDICT WATER DEMAND. That system feeds raw probe readings straight into its demand model, which is exactly the gap this smoother fills.

The evaluation. Real field data has no clean ground truth, so we score the smoother in three ways that avoid needing one. The first is timing. On NEON records we measure the lag between recorded precipitation and the smoother's declared wetting onset. The second is consistency. We check agreement with offline PELT segmentation, the non-causal changepoint method used in the NEON paper, treated as an upper reference rather than as truth. The third is stability. We measure how much the causal drydown-rate estimate varies within segments that PELT identifies as a single drying episode. This follows the spirit of Mixture to Mixture: Leveraging Close-talk Mixtures as Weak-supervision for Speech Separation, which trains separators on real recordings with only imperfect references instead of synthetic clean targets. We apply the same stance to evaluation rather than training.

The simulation test. We build a closed-loop soil-water simulator calibrated on field records in the manner of Optimizing Irrigation Efficiency using Deep Reinforcement Learning in the Field. We run it under two noise regimes. One adds Gaussian noise. The other replays real residuals: what remains after PELT-fitted drydowns are subtracted from NEON and field records, with their daily structure and timing preserved. Every smoother then drives the same threshold controller in both regimes. This part directly tests the concern raised about Performance triggered adaptive model reduction for soil moisture estimation in precision irrigation, whose 90 virtual sensors receive noise only from the simulation itself.

Baselines

Do nothing. The naive reference is the raw probe stream driving the threshold controller directly, as in the Raspberry Pi system cited above. Every other method must beat it on false trigger crossings, or smoothing is pointless for this control task.

Trailing moving average and causal Savitzky–Golay. These are the classical references. The trailing moving average uses a window width grid-searched on held-out NEON data. The causal Savitzky–Golay filter has its window and polynomial order grid-searched the same way, echoing the search in Annual Cropping Intensity Dynamics in China from 2001 to 2023. To keep the comparison fair, every method gets the same tuning budget, the same data splits, and the same access to the temperature channel. We also add a moving average applied after linear temperature regression, so that the temperature term alone is credited separately.

Switching Kalman filter (strongest competitor). This filter uses an exponential-decay process model, an added jump process for wetting events, and a measurement noise level estimated adaptively. It encodes the same physics as our smoother in the most established form, so it is the comparison that decides whether our specific construction adds anything. Its noise parameters get the same tuning budget as ours.

Offline PELT. This is the non-causal oracle and the upper bound on onset timing. It shows how much of the gap to perfect segmentation our causal method closes.

Ablations. Two ablations isolate our key ingredients: removing the wetting-reset (a drydown-basis fit in a plain sliding window), and removing the temperature term.

What counts as winning. No reported result was found on Lacuna for any smoother evaluated on NEON soil-moisture streams, so we set targets in advance. The smoother beats a baseline if, at matched residual variance during drydowns, it lowers median wetting-onset lag and cuts false threshold crossings, with confidence intervals from a block bootstrap over sites and seasons. For the closed-loop experiment, the strongest reported reference point is DRLIC's in-field water saving of 9.52% over ET-based control and 3.79% over a sensor-threshold method (Optimizing Irrigation Efficiency using Deep Reinforcement Learning in the Field). We do not claim to beat DRLIC. We ask whether better sensing narrows the gap between a simple threshold controller and that learned controller.

Why not just…?

Why not just use a switching Kalman filter? This is the real threat. A well-tuned jump-plus-decay Kalman filter may match our smoother, since it uses the same physics. We include it as the main baseline rather than arguing it away. If it matches us, the method contribution shrinks to a cheaper, easier-to-tune equivalent. The paper's weight then rests on the simulation-versus-real-noise ranking result, which still holds and is still new. Our expectation that the windowed fit helps rests on one specific weakness: a single noise covariance adapts slowly to the daily temperature artifact, which is not white noise. The temperature-term ablation, applied to both methods, will show whether that is the real difference.

Why not just widen the moving-average window, or use a median filter? The evidence against this is that widening the window buys noise suppression at the cost of blurred transitions and attenuated peaks, as The Application of FFT Analysis on the Feeder Current Based on the Smooth Function in MATLAB shows. For a trailing window, that blur shows up as onset delay. A median filter handles isolated spikes but not a smooth daily oscillation. Both are in the grid search, so the onset-lag versus false-crossing trade-off curve will show directly whether any window width reaches our operating point.

Why not just add hysteresis to the controller and skip smoothing? Soil moisture changes slowly, and a dead-band around the trigger might absorb the noise. We treat this as a serious alternative and add threshold-with-hysteresis as an extra controller baseline. The hot-pepper trial in Precision Drip Irrigation on Hot Pepper in Arid Northwest China Area found peak water use efficiency of about 11.5 kg/m³ when triggering between −30 and −40 kPa, 8.3 kg/m³ at −10 kPa, and a 16% yield drop at −50 kPa. A dead-band wide enough to absorb the daily swing may push the effective trigger out of that productive band. If hysteresis matches our smoother in closed-loop water use and time spent in the band, we will report that smoothing is unnecessary for threshold control.

Proposed Experiments

Offline real-data study. The first experiment uses the NEON hourly soil moisture and precipitation records at SERC, SRER and TALL used in A changepoint approach to modelling nonstationary soil moisture dynamics. We train on some seasons and test on others. We report three scores: wetting-onset lag against precipitation timestamps, agreement with PELT segments, and stability of the causal drydown-rate estimate within segments. We report the paper's own simulation figure only as context: changepoint true positive rates "over 90% in many scenarios."

Decisive simulation experiment. For every smoother, we compute its rank on false trigger crossings and irrigation water use in the calibrated simulator, once under Gaussian noise and once under replayed real residuals. We then compute the rank correlation between the two orderings. A low correlation, or a reversal between the moving average and our smoother, is the evidence that simulation-only evaluation misleads in this domain. This parallels the finding in Ultra-Low Latency Speech Enhancement -A Comprehensive Study that methods which work in simulation lose to filtering baselines on real blind test sets. To connect the result to plant stress, we convert the trigger crossings to the critical-moisture framing of Global critical soil moisture thresholds of plant water stress, where stress onset ranges from 0.12 m³/m³ in arid zones to 0.26 m³/m³ in humid zones.

Small field check. The last experiment is a small drip plot with paired co-located probes and a soil-temperature sensor at 0.2 m depth. It tests whether the temperature coefficient $\beta$ fitted on one probe transfers to its neighbour, and whether the smoothed stream keeps the controller inside the −30 to −40 kPa band more often than the raw stream. Converting probe moisture readings to soil water potential needs a site-specific soil-water retention curve, which we will measure rather than assume.