Reading Thermal Crossover from the Slope of Temperature, Not Its Level

A thermal camera sees objects because they are warmer or colder than what surrounds them. Twice a day, near sunrise and late afternoon, an object and its background pass through the same temperature. For a short window the contrast is zero, the object disappears from the image, and detectors and trackers fail. This failure is called thermal crossover. Our claim is that the object has not actually become indistinguishable at crossover. Objects made of different materials heat and cool at different rates, so at the moment their temperatures are equal, their temperatures are still changing at different speeds. A detector that reads each pixel's recent temperature history and measures how fast it is changing, rather than only how hot it is, should therefore keep working through crossover. It needs no second camera.

The idea comes from rare-event physics. There, detectors face the same situation: a background event deposits the same energy as the signal, so amplitude alone cannot separate them. Pulse-shape discrimination solves this by reading the time profile of the event instead of its height. In germanium detectors, surface alpha particles are rejected by the slow tail of their charge pulse. In liquid argon, nuclear recoils are separated from electron recoils by the fraction of light that arrives promptly. We bring two things from that field into thermal vision. The first is a temporal, gain-normalized "prompt fraction" feature computed per pixel over minutes of video. The second is the field's way of evaluating: report how much background leaks through at a fixed signal acceptance, measured specifically inside the zero-contrast window.

Core Claim

Model each surface as a first-order thermal body that relaxes toward the ambient temperature $T_a(t)$ with time constant $\tau$, where $\tau$ is set by heat capacity and coupling to the air. The object–background contrast $C(t) = T_o(t) - T_b(t)$ then evolves as

$$ \frac{dC}{dt} = \frac{T_a - T_o}{\tau_o} - \frac{T_a - T_b}{\tau_b}. $$

At crossover $T_o = T_b = T$, so this reduces to $dC/dt = (T_a - T)(1/\tau_o - 1/\tau_b)$. This is nonzero whenever the two materials have different time constants and ambient temperature is still moving, which is exactly the situation at dawn and dusk. In words, crossover is a zero crossing of contrast, and the slope at that crossing carries the information the level has lost.

We operationalize this with a per-pixel two-window ratio, analogous to the prompt light fraction:

$$ F(x,t) = \frac{T(x,t) - T(x,t-w_s)}{T(x,t) - T(x,t-w_l)}, $$

where $T(x,t)$ is the radiometric value of pixel $x$ at time $t$, $w_s$ is a short look-back window, and $w_l$ is a long one. Like the prompt fraction, $F$ is invariant to sensor gain and to the absolute temperature level, so it should transfer across days with different weather. We pair $F$ with a raw rate channel and a per-pixel fitted $\tau$. These are added as extra input channels to a standard detector.

The novelty is specific. No paper in our search computes temporal heat-exchange kinetics for crossover. The closest work, Open Dataset and Enhancement Method for Long-wave Thermal Diurnal Material Classification, shows that materials follow distinct diurnal temperature paths. It handles crossover with two single-frame spatial statistics: Michelson contrast against the surroundings and internal texture standard deviation. We keep their setting but replace spatial statistics with temporal ones. We also make a falsifiable prediction they do not: our gain over their method should be largest exactly where their contrast channel goes to zero.

Research Program

The primary testbed is the Diurnal Material Dataset from that same paper. It contains 72,703 long-wave thermal frames, taken every 10 seconds over 1.5 months, of six materials (aluminum, rubber, glass, steel, Styrofoam, wood) against a rock background, with crossover windows at roughly 7–8 AM and 3–5 PM. The 10-second cadence over a static scene is what makes per-pixel histories meaningful. This also bounds the scope honestly: the claim is about fixed or registered cameras watching scenes over minutes, such as surveillance, monitoring, and infrastructure inspection.

We will build three things. First, the temporal channels $F$, the rate $dT/dt$, and a sliding-window estimate of $\tau$ obtained by regressing pixel temperature on a smoothed ambient proxy. The ambient proxy is taken from background pixels, so no external weather sensor is needed. Second, a YOLOv11 detector that takes these channels together with thermal intensity, matching the detector used by the diurnal-dataset paper. Third, an analysis that bins test frames by measured spatial contrast and reports performance per bin. This tests whether the temporal channels rescue frames precisely where spatial contrast fails.

Splits will be by time rather than random. We train on the earlier weeks and test on the later weeks. Random frame splits at 10-second cadence would leak nearly identical neighbouring frames between train and test, and a temporal holdout also probes generalization across changing weather.

A secondary study uses the surveillance-scenario sequences of LSOTB-TIR: A Large-Scale High-Diversity Thermal Infrared Object Tracking Benchmark and the crossover-attributed sequences of PTB-TIR: A Thermal Infrared Pedestrian Tracking Benchmark. These are video-rate sequences only seconds to minutes long, so the diurnal relaxation mechanism does not apply directly. We predict the temporal channels will help little on hand-held, drone-mounted, and vehicle-mounted sequences, where ego-motion destroys pixel histories. On static surveillance cameras, a short-window version restricted to background estimation may still help. If the method helped equally on moving-camera clips, that would suggest it is exploiting motion rather than thermal kinetics. The scenario split is therefore a diagnostic, not a headline.

Baselines

The naive reference point is the thermal-only YOLOv11 trained on the Diurnal Material Dataset. Open Dataset and Enhancement Method for Long-wave Thermal Diurnal Material Classification reports an overall F1 of 94.6 for this model, falling to 90.3 during crossover events. We will re-run it on our temporal split so the comparison is not distorted by a different split.

The strongest existing method is the same paper's thermal-plus-contrast model. It feeds Michelson contrast and internal contrast visibility as extra channels and reports a crossover F1 of 93.5 and an overall F1 of 95.5. It is the right competitor because it is the only published single-sensor method that targets diurnal crossover on this exact data. Fairness requires the same architecture, input resolution, training epochs, augmentation, and hyperparameter search budget. The only difference is which extra channels are supplied. We will also run the combination of their spatial channels with our temporal channels, since the two are complementary in principle.

Beating this baseline means a crossover-window F1 above our re-run of their model, with a confidence interval from day-level bootstrap resampling that excludes zero improvement, and no loss in overall F1.

The most important ablation is a learned temporal baseline that receives the same raw history as our features. It stacks the frames at $t$, $t-w_s$, and $t-w_l$ as raw channels, or more frames up to the same total look-back, and lets the network learn whatever temporal function it wants. This isolates whether our gain comes from access to temporal information or from the physically normalized form of the feature. Further ablations drop $F$ and keep only the raw rate, drop the fitted $\tau$, and sweep $w_s$ and $w_l$. The thermal model predicts that useful windows sit near the materials' time constants.

RGB-thermal fusion trackers such as HMFT from Visible-Thermal UAV Tracking: A Large-Scale Benchmark and New Baseline cannot be run on this thermal-only dataset. We discuss them below rather than compare against them directly.

Why not just…?

Why not just stack several past frames and let the network learn the temporal cue? This is the most serious alternative, and we do not assume it loses. A network given the right frames could in principle learn a temporal difference. Our bet is narrower. A gain- and level-normalized ratio should generalize better when the test weeks have different ambient ranges than the training weeks, much as the prompt fraction in First Results from the DEAP-3600 Dark Matter Search with Argon at SNOLAB separates event classes without depending on event energy. The matched-history ablation, evaluated on the temporal holdout and with reduced training data, settles this directly. If raw stacking matches our features under both conditions, we will report that the contribution is the temporal-history observation itself, not the feature design.

Why not just add a visible camera? RGB-thermal trackers such as SGT from RGB-T Object Tracking: Benchmark and Baseline and HMFT handle crossover by shifting weight to the visible stream. Yet the VTUAV authors still report thermal crossover among the hardest attributes. Diurnal crossover also falls at dawn and dusk, when visible light is weakest. A second camera adds aligned hardware that single-aperture thermal deployments lack. We see temporal kinetics as complementary to fusion, not a replacement for it.

Why not just use richer spatial texture, such as HOG, LBP, or internal standard deviation? These help. HOG and LBP gave roughly 20% precision gains over raw intensity on PTB-TIR, and internal contrast lifted crossover F1 from 90.3 to 93.5. However, they are computed within a single frame, and they homogenize together with mean contrast once surfaces equilibrate. Our contrast-binned analysis tests this directly. If spatial channels already recover the lowest-contrast bins as well as temporal channels do, our mechanism adds nothing there.

Why not just train bigger thermal models on more data? LSOTB-TIR: A Large-Scale High-Diversity Thermal Infrared Object Tracking Benchmark shows that thermal-specific training helps substantially; SiamFC-TIR gained 13.5% precision on PTB-TIR. Still, after training on 1,400 sequences the authors name crossover the most critical remaining challenge. More data does not create information that a single frame lacks. Adaptive spatial regularization such as RAMCT: Novel Region-adaptive Multi-channel Tracker with Iterative Tikhonov Regularization for Thermal Infrared Tracking improves tracking (81.7% precision on LSOTB-TIR) at 13 frames per second. It too operates on spatial features that flatten at equilibrium.

Decisive Checks

Three results would decide the proposal. First, in the lowest spatial-contrast bin of the temporal holdout, temporal channels should beat the spatial-contrast channels of the diurnal-dataset paper. The gap should shrink as contrast grows; a flat gap across bins would mean the gain is not tied to crossover.

Second, the normalized ratio should beat matched raw-frame stacking on held-out weeks. If it does not, the feature-design claim fails.

Third, borrowing the rare-event reporting convention, we fix material recall at 95% inside crossover windows and report the false-detection rate on the rock background there. This number tells a practitioner how much clutter survives at a usable operating point, which a single F1 hides.

As a sanity check on the physics, materials whose fitted $\tau$ values are closest to the rock's should show the smallest temporal gain. If gains do not track $\tau$ differences, the mechanism we describe is not what the network is using.