Doctors grading a chronic wound or burn do not just look at the open wound. They check the warmth and redness of the skin around it, because inflammation and poor blood flow in that rim separate a stable stage from one that is getting worse. Medical imaging models usually throw that information away. To stop models from latching onto irrelevant background, the standard fix is to segment the wound and keep only the pixels inside it. That fix is reasonable for an ordinary color (RGB) photograph. For a thermal camera it is the wrong choice, because the thermal signal that matters sits mostly outside the visible wound edge. We propose a two-sensor wound-staging model built on this asymmetry. A color photograph supplies a sharp wound outline. A paired thermal image is then read only as a short, physically normalized temperature profile in rings around that outline, compared against the patient's own healthy skin. A small reliability check switches the thermal input off when the wound and its surroundings are at the same temperature.
The claim is narrow on purpose. For the hard distinctions in wound staging, the boundaries between adjacent severity grades, most of the useful extra information from thermal imaging lies in the peri-wound rim. A coarse, low-dimensional ring profile captures it better than either a mask copied from the color image or a deep network applied to the whole thermal image. If the claim holds, the gain should show up mainly in confusions between adjacent grades, not in healthy-versus-diseased detection. It should also survive the main practical weaknesses of thermal cameras: they see much less detail than color cameras, they are physically offset from them, and their contrast disappears when temperatures equalize.
What the literature establishes and what it leaves open
Three findings motivate the proposal.
First, models trained on whole images exploit irrelevant regions. The Role of Pleura and Adipose in Lung Ultrasound AI showed that a lung-ultrasound severity model relied on the thickness of the fat layer under the skin. Masking that layer out raised AUC from 0.6553 to 0.7742. Two other papers move the mask into the network instead of the image. Computed tomography–based COVID–19 triage through a deep neural network using mask–weighted global average pooling averages network features only over lung tissue. Lesion-Centric Latent Phenotypes from Segmentation Encoders for Breast Ultrasound Interpretability does the same over the predicted lesion and reports a malignancy-probing AUC of 0.982, against 0.852 for an unmasked CNN. All three use one sensor and one region. The research file also names the cost of this approach: hard masks discard the transition zone at the lesion border, where clinical signs often appear. We take that limitation as our starting point.
Second, fine-grained staging is where single-sensor models fail. In A Framework for Automatic Burn Image Segmentation and Burn Depth Diagnosis Using Deep Learning, the model separates burn from no-burn well (IoU 0.8467), but its mean IoU across burn depths falls to 0.5144. The authors attribute this to the visual similarity of deep partial-thickness and full-thickness burns in RGB images. LUTSeg: A Longitudinal Multi-Expert Dataset for Ulcer Tissue Segmentation reports near-zero expert agreement on some internal tissue classes. Its TiSage method reaches 31.70 mIoU with one-eighth of the labels, using RGB images only. Fine-Grained Classification of Cervical Cells Using Morphological and Appearance Based Convolutional Neural Networks shows how accuracy decays as the grading gets finer, from 94.5% (2 classes) to 71.3% (4 classes) to 64.5% (7 classes), even when ground-truth masks are added as extra input channels. That paper also supplies the simplest mask-as-input design, which we use as a baseline.
Third, thermal imaging adds physiology but has known physical failure modes. Holistic multi-class classification & grading of diabetic foot ulcerations from plantar thermal images using deep learning grades diabetic foot ulcers on six levels from thermal images alone. It corrects for ambient temperature drift with a Thermal Change Index, a temperature measure normalized per foot artery region against healthy baselines. Its CNN still sees the whole foot, and there is no color photograph to fall back on when thermal contrast vanishes. Methods that fuse color and thermal images, such as ABMDRNet: Adaptive-Weighted Bi-Directional Modality Difference Reduction Network for RGB-T Semantic Segmentation, DIDFuse: Deep Image Decomposition for Infrared and Visible Image Fusion, and SGDFuse: SAM-Guided Diffusion Model for High-Fidelity Infrared and Visible Image Fusion, assume pixel-aligned images and coarse categories such as cars and pedestrians. They treat thermal as a second image to be merged pixel by pixel. We instead treat it as a low-resolution physiological measurement taken at a location chosen by the color image.
The research file contains no work that combines a color-derived wound outline, thermal measurements read off-lesion, and ordinal staging. The closest pair-wise works differ in specific ways. Muralidhara et al. use thermal imaging with granular grades but have no color sensor and no wound-relative geometry. LUTSeg and the burn framework use color images with granular labels but have no thermal sensor. ABMDRNet fuses the two sensors but has no clinical grading.
Core Claim
Let $M$ be the wound mask predicted from the color image. Let $T$ be the thermal image after it has been coarsely aligned to the color image. For each band $k$ we form a ring $R_k$ of skin between distances $d_{k-1}$ and $d_k$ outside the wound edge. The peri-wound thermal profile is
$$ p_k = \frac{\operatorname{median}_{x \in R_k} T(x) - \mu_{\text{ref}}}{\sigma_{\text{ref}}}, $$
where $\mu_{\text{ref}}$ and $\sigma_{\text{ref}}$ are the mean and spread of temperature on a reference patch of healthy skin, taken from the contralateral foot or limb when available and otherwise from distant skin on the same side. The median and the ring geometry make $p$ insensitive to alignment errors of a few pixels and to the thermal camera's low resolution. Normalizing against the patient's own reference skin removes ambient drift, which plays the same role as the Thermal Change Index.
A reliability score $r$ is the absolute skin-to-ambient contrast relative to $\sigma_{\text{ref}}$. It gates the profile, so the final ordinal predictor sees $[f_{\text{RGB}}(M),\, g(r)\, p]$. Here $f_{\text{RGB}}$ is the color-image representation averaged over the wound mask, and $g$ is a learned gate trained with random thermal dropout. When temperatures equalize and $r$ is small, the model falls back to color.
We make three predictions. Adding $p$ should reduce adjacent-grade errors substantially more than it changes binary detection. A profile restricted to the wound interior should recover little of that gain. The gated model should never do worse than the color-only model, even in cases where thermal contrast has collapsed.
Research Program
The work has two parts.
The first part tests each component on existing single-sensor benchmarks. On LUTSeg and DFUTissue, we check that the color-image wound outline is accurate enough at the outer margin, where LUTSeg reports high expert Dice. We also measure how mask errors propagate into the ring locations, which addresses the error-propagation problem the research file flags.
On the plantar thermal dataset of Muralidhara et al., we test the thermal half alone. The comparison is a whole-foot CNN, scalar Thermal Change Index features, and ring profiles around annotated lesions where lesions exist. We use patient-level splits. The reported 98.27% accuracy relies on augmentation up to 400 images per class, so that number is likely saturated or inflated by images of the same patient appearing on both sides of the split, and it cannot discriminate between methods unless we re-split by patient.
On MFNet, we stress-test the gate. Its day and night pairs, combined with synthetic temperature equalization (compressing thermal contrast toward the background), let us check whether $g(r)$ shuts off a useless thermal input without hurting color-only performance.
The second part, which carries the main result, is a paired clinical cohort. We would collect diabetic foot ulcers and burns with a color camera and an attached low-resolution longwave thermal camera. Each case gets an ordinal grade (a Wagner-type ulcer grade, and burn depth across the four classes S/ST/DT/FT) from at least three raters, so that model error can be compared with the spread between raters. Ambient room temperature is logged for every capture.
We deliberately do not build a deep thermal backbone. Thermal images lack texture and are typically 160×120 to 320×240 pixels, so a profile of six to ten numbers is closer to what the physics supports.
Baselines
The naive reference is a color-only classifier on the full image. A second reference is the same classifier with wound-mask pooling, which is the single-sensor masking result of Gare et al. and of the breast-ultrasound latent-phenotype work applied to wounds. All models share one pretrained color backbone, the same patient-level folds, the same augmentation, and an equal hyperparameter search budget of a fixed number of trials per method on the validation fold.
The strongest existing fusion competitor is an ABMDRNet-style network that translates and fuses features between the two sensors, given the same thermal input upsampled and aligned to the color image. On MFNet it reports 54.80% mIoU, and about 51% without its bidirectional translation module. We first reproduce roughly that number on MFNet to verify our implementation. We then retrain it on the clinical cohort with a classification head.
The strongest thermal-grading competitor is the Muralidhara et al. pipeline: Thermal Change Index features plus a whole-thermogram CNN with aspect-ratio-preserving inputs. It reports 98.27% mean accuracy, 0.9684 sensitivity, and 0.9892 specificity on its plantar dataset. We re-run it with patient-level splits, both on that dataset and on the thermal half of our cohort. For RGB tissue segmentation, the reference is TiSage at 31.70 mIoU on LUTSeg with one-eighth of the labels. For burn depth in RGB, the reference is HRNetV2 at a mean IoU of 0.5144. There is no reported result on Lacuna for any method on paired color-thermal wound grading, so the cohort numbers will be new.
Three ablations isolate the key ingredient. The first computes the profile inside the wound mask instead of outside it. The second replaces rings with a single region-wide mean, which amounts to a Thermal Change Index scalar. The third removes the gate.
Beating the baselines means two things. The full model should improve quadratic weighted kappa over both the color-only masked model and the ABMDRNet-style fusion model, with a 95% patient-bootstrap interval excluding zero. The gain should also be concentrated in the adjacent-grade confusion cells: DT versus FT, and neighboring ulcer grades.
Why not just…?
Why not just stack color, thermal, and the mask into one input image, as Lin et al. did with nucleus and cytoplasm masks? This is the cheapest alternative and a real one. Its weakness is that it requires pixel alignment between two cameras with different optical axes and resolutions. The research file identifies parallax and resolution mismatch as causing doubled edges when images are fused pixel by pixel. We include this design as a baseline and test it under injected misalignment of 2, 4, and 8 pixels. If it matches the ring-profile model at zero misalignment and degrades only gently under shift, we will report that the off-lesion geometry, not the pooling, is the contribution.
Why not just train a strong fusion network on the whole image and let it find the peri-wound rim on its own? It might find the rim. Gare et al. show it might equally find body habitus or bedding. The Yap et al. evaluation of foot-ulcer detection (Deep Learning in Diabetic Foot Ulcers Detection: A Comprehensive Evaluation) notes false positives triggered by toenails, skin folds, and bedding. We test this directly with a background-swap probe: pasting each wound onto other patients' surroundings. A large prediction shift under swapping, for the whole-image model but not for ours, would show shortcut use even if accuracies match.
Why not just add Thermal Change Index scalars to a color classifier? This is the most serious threat, because the ring profile is itself a hand-built physical feature. The ring-versus-region-mean ablation settles it. If the scalar matches the profile, the honest conclusion is that thermal adds value in some simple form, and that the radial structure does not matter.
Proposed Experiments
The decisive check is a breakdown of the confusion matrix on the paired cohort. The prediction is that the ring-profile model reduces adjacent-grade errors by more than it changes binary sensitivity. A within-mask profile and a whole-image fusion model should not show this pattern.
A second check stratifies results by logged ambient contrast. In the lowest-contrast tercile, the gated model should equal the color-only model. In the highest tercile, it should exceed it. An ungated model should fall below color-only in the low-contrast cases.
A third check trains with masks drawn from our own segmenter, not expert masks, and varies mask erosion and dilation. This measures how mask errors move the rings and how much performance that costs.
All results are reported as quadratic weighted kappa, mean absolute grade error, and per-adjacent-pair accuracy. Each is placed alongside the agreement between raters, because on classes where experts disagree, as in LUTSeg, a model cannot be credibly scored above the human ceiling.
References
- The Role of Pleura and Adipose in Lung Ultrasound AI
- Computed tomography–based COVID–19 triage through a deep neural network using mask–weighted global average pooling
- Lesion-Centric Latent Phenotypes from Segmentation Encoders for Breast Ultrasound Interpretability
- A Framework for Automatic Burn Image Segmentation and Burn Depth Diagnosis Using Deep Learning
- LUTSeg: A Longitudinal Multi-Expert Dataset for Ulcer Tissue Segmentation
- Fine-Grained Classification of Cervical Cells Using Morphological and Appearance Based Convolutional Neural Networks
- Holistic multi-class classification & grading of diabetic foot ulcerations from plantar thermal images using deep learning
- ABMDRNet: Adaptive-Weighted Bi-Directional Modality Difference Reduction Network for RGB-T Semantic Segmentation
- DIDFuse: Deep Image Decomposition for Infrared and Visible Image Fusion
- SGDFuse: SAM-Guided Diffusion Model for High-Fidelity Infrared and Visible Image Fusion
- Deep learning in diabetic foot ulcers detection: A comprehensive evaluation