Nuisance-Discounted Similarity Graphs for Lab-to-Field Plant Disease Transfer

Plant disease classifiers trained on clean laboratory photographs, meaning single leaves on plain backdrops under steady light, score near-perfectly at home and fail in real fields. Labeled field images are scarce, and unlabeled ones are plentiful. A natural remedy is to connect the two collections with a similarity graph: link each image to the images it most resembles in a pretrained feature space, then let disease labels flow along the links from labeled lab images to unlabeled field images. Our diagnosis is that this remedy fails in a specific way. In a generic image embedding, the most similar image is usually one taken under the same conditions, with the same backdrop, lighting, or blur, and not one showing the same disease. The graph therefore splits into a lab island and a field island, each organized by capture conditions, and labels have almost no path from one to the other.

We propose to build the graph differently. For each pair of images, we measure how much of their similarity is explained by shared capture conditions and subtract that part before choosing neighbors. We then require every field image to keep at least a few links into the lab collection. Labels are propagated over this "nuisance-discounted" graph. The claim is narrow and testable: removing condition-driven similarity before sparsifying the graph raises the fraction of edges that join same-disease images, and that should raise unlabeled field accuracy over propagation on a plain nearest-neighbor graph. If cropping images to the leaf gets the same benefit, then the simpler fix wins. We designed the experiments so that outcome would be visible.

Core Claim

Work on semantic similarity graphs, meaning graphs whose nodes are items and whose edges connect items with nearby embeddings, has shown that the way edges are chosen matters more than the propagation algorithm run on top. Scale-Free Graph-Language Models builds a cosine k-nearest-neighbor graph over text features and uses a graph neural network to spread labels from 50 labeled nodes. It reaches 32.51% on ogbn-arxiv, against 16.61% for a GCN+DeBERTa baseline, and the authors stress that the choice of cosine similarity is essential. LGDE replaces plain k-NN with continuous k-NN (CkNN), which adapts each node's neighborhood radius to local density, and adds random-walk diffusion. On a 4chan corpus it finds 30.2% relevant terms against 18.9% for cosine thresholding. These two papers supply our method: they show that a well-built graph lets supervision travel along multi-hop chains that direct pairwise matching misses. Both work only on text, where "nuisance" similarity is mild.

The closest idea to ours is in GraphGPT-Patent. That work damps edges in a patent similarity graph when they cross technology domains or large time gaps, because boilerplate and vocabulary drift create false links. Its principle is that edge weights should be conditioned on known sources of spurious similarity. Copying its rule directly into our setting would backfire. Down-weighting cross-domain edges would cut exactly the lab-to-field links we need. We invert the rule. We damp edges in proportion to how much of their similarity comes from shared capture conditions, which usually means within-domain edges. Cross-domain edges that survive this discount are the ones carrying disease information.

The vision papers motivate the problem and set its constraints. PlantSeg shows how severe the lab-to-field collapse is: MobileNetV3 reaches 10.22% mIoU and even ConvNeXt-L only 46.24% mIoU on in-the-wild lesions. The same paper explicitly calls for affinity-based modeling. Adaptive Attention Distillation documents "attention drift," where perturbations pull a model's focus onto background textures. That is the image-level version of the edge corruption we target, but the paper handles it only between isolated support and query pairs, never across a dataset-wide graph. Mask-RCNN-CBAM reports a roughly 4.2% accuracy drop in foggy, low-contrast fields even with attention modules. We take this as evidence that stronger attention inside the network does not by itself remove sensitivity to capture conditions.

Research Program

Nodes are images from a labeled lab source and an unlabeled field target. Each image $i$ gets a frozen semantic embedding $z_i$ from a pretrained vision encoder, and a nuisance descriptor $u_i$ that deliberately ignores the disease. We build $u_i$ in two ways. The first embeds the image with the central leaf region masked out, leaving only the background. The second is a short vector of capture statistics: mean brightness, contrast, variance of the Laplacian as a blur measure, and a color histogram of the image border.

For each pair we compute the semantic cosine similarity $s_{ij}=\cos(z_i,z_j)$ and the nuisance similarity $n_{ij}=\cos(u_i,u_j)$. We then fit a single coefficient $\beta$ by least squares over a random sample of pairs and define the discounted similarity

$$r_{ij} = s_{ij} - \beta\, n_{ij},$$

which is the part of semantic similarity that shared conditions do not predict. The regression form is the simplest estimate of "similarity explained by nuisance." A nonlinear version, a small regressor on the vector of nuisance similarities, is a planned extension if the linear one underfits.

The graph is a CkNN graph on $r$, following LGDE's construction, with one additional constraint: a cross-domain quota. Every field node keeps its top $q$ lab neighbors under $r$, even if they fall outside its CkNN neighborhood, so the graph cannot split into a lab island and a field island.

Labels are spread in two stages. The first is closed-form label propagation: iteratively average neighbors' label distributions with the lab labels clamped. The second, following the Scale-Free Graph-Language Models recipe, trains a small graph neural network on the propagated labels, keeps its confident predictions as pseudo-labels, and fine-tunes a lightweight classifier on lab labels plus field pseudo-labels. That last step matters for deployment, because it produces a standalone model that needs no graph at test time.

Alongside accuracy, we will report graph diagnostics that make the mechanism inspectable, computed using held-out field labels that are never used in training. Edge purity is the fraction of edges joining same-disease images. Cross-domain edge fraction is the share of edges linking lab and field images. Nuisance share is the correlation between edge weight and $n_{ij}$.

Baselines

The naive reference point is a classifier trained only on the lab source with standard augmentation (flips, rotations, color jitter) and applied directly to field images. This follows the MobileNetV2 recipe of Plant Leaf Disease Detection using MobileNetV2, which reports 94.32% on PlantVillage. Its field accuracy defines the gap. Beating it is necessary but not interesting.

The strongest competitor is label propagation, followed by SFGL-style GNN pseudo-labeling, on a plain cosine k-NN graph over the same frozen features, as in Scale-Free Graph-Language Models. It is the strongest because it already exploits unlabeled field data and multi-hop transfer, so it isolates exactly what our edge rule adds. On our task: no reported result found on Lacuna for graph propagation from PlantVillage to PlantDoc, FGVC8, or any field plant dataset. The only reported number for the recipe is 32.51% versus 16.61% on ogbn-arxiv at 50 labels. We will run it with the same encoder, the same neighbor count range, the same propagation and fine-tuning schedule, and an equal hyperparameter search of 20 configurations per method, selected on a small labeled field validation split identical across methods. Beating it means a field macro-F1 gain whose 95% bootstrap interval over three seeds excludes zero, together with a measurable rise in edge purity. An accuracy gain without a purity gain would mean our explanation is wrong.

Two further strong baselines receive the same budget. The first is confidence-thresholded self-training, which pseudo-labels field images using the lab classifier's own confident predictions with no graph. The second is the few-shot adaptation approach of Adaptive Attention Distillation, given the same handful of labeled field images per class. That paper reports a 3.3% to 8.75% average mIoU improvement over prior few-shot methods on ER-FSS but no number on our splits.

As supervised upper references we quote FoldNet at 90.49% accuracy on FGVC8 and TomFormer at 81% mAP on PlantDoc from TomFormer. Both were trained on full field labels and are not directly comparable. They mark the ceiling that zero-label transfer approaches.

The ablations isolate our key ingredient. We run the method with $\beta=0$, which keeps the CkNN construction and the quota but removes the discount. We run it without the quota. We run it with the discount but using per-domain mean-centering of embeddings in place of the continuous nuisance regression.

Why not just…?

Why not just crop to the leaf and use plain k-NN? If capture-condition similarity lives almost entirely in the background, a leaf segmenter followed by an ordinary graph could match our method with less machinery, and this is the most serious threat. Lighting, blur, and fog also act on the leaf pixels themselves, which the 4.2% fog drop in Mask-RCNN-CBAM suggests, so we expect cropping to fix backdrop-driven edges but not illumination- or blur-driven ones. We run leaf-cropped k-NN propagation as a full baseline and also stack our discount on cropped features. If cropping alone matches us, we will report that the simpler method is sufficient.

Why not just subtract each domain's mean embedding? Per-domain centering is a one-line fix and removes the gross lab-versus-field offset. It treats every field image as sharing one condition, whereas field collections mix sun, shade, handheld phones, and greenhouse lighting, as in KUTomaDATA. The centering ablation measures this directly. The continuous regression must beat it specifically on field subsets stratified by capture statistics, or the added complexity is not justified.

Why not just use a stronger attention backbone or scale training data? WildDash shows that training on the larger, more diverse Mapillary Vistas still leaves a model at 30.7 mIoU under hazards, against roughly 64 in-domain. PlantSeg shows that ConvNeXt-L tops out at 46.24% mIoU. Neither result rules out a much larger encoder. For that reason we repeat the whole comparison with two encoders of different size. If the gap between our method and plain k-NN shrinks toward zero as the encoder grows, the discount is a patch for weak features rather than a lasting contribution, and we will say so.

Proposed Experiments and Decisive Checks

The source domain is PlantVillage: 54,305 lab images in 38 classes. The targets are PlantDoc, restricted to plant-disease classes shared with PlantVillage; FGVC8, restricted to its apple classes that overlap PlantVillage; and the field maize data of Qian et al., restricted to its three diseases plus healthy. Each target is split into an unlabeled pool, a small labeled validation split used only for model selection, and a held-out test set. We report test macro-F1 and accuracy on shared classes, plus the three graph diagnostics.

The first decisive check is mechanistic. On the plain k-NN graph we expect a low cross-domain edge fraction and a strong correlation between edge weight and $n_{ij}$. Our construction should raise edge purity and cross-domain fraction together. Across all methods and targets, field accuracy should track edge purity. If purity rises but accuracy does not, graph quality is not the bottleneck.

The second check is controlled. We apply synthetic fog, blur, and overexposure to a random half of the field pool and add the corresponding statistics to $u_i$. Plain k-NN should then form hazard-driven clusters, meaning fogged images linking to fogged images. Our discount should dissolve them, and the accuracy gap between our method and plain k-NN on the hazard-corrupted half should exceed the gap on the clean half.

The third check, a stretch goal, moves from images to lesion patches on PlantSeg. There nodes are image patches, and the same discounted graph propagates lesion labels among fragmented regions. This directly answers the PlantSeg authors' call for affinity modeling of small, non-convex lesions, measured by mIoU on lesions covering under about 1% of the image against the 46.24% overall reference.