Painting Lesions onto Healthy Field Leaves for Few-Shot Crop Disease Detection

Crop disease scouting is slow because a person has to walk the rows. The obvious replacement, a detector running on a phone, robot, or drone, is slow for a different reason: someone must draw boxes around hundreds of lesions per disease, in real field conditions, often during the short window when a disease is visible. The two kinds of data are very unequal in cost. Images of healthy plants in the field are nearly free, since any robot pass collects thousands. Annotated images of a specific disease in the field may number in the single digits. This proposal uses that asymmetry. A generative model learns from a handful of real annotated lesions and paints new lesions onto the abundant healthy field images. The model is an inpainting diffusion model, meaning a generator that fills a masked region of a real photo by gradually denoising random noise until it matches the surrounding image. The detector then trains on these composites as if they were labeled field data.

The obvious version of this idea has a known failure mode, and the proposal is built around it. Work in medical imaging shows that downstream models are very sensitive to whether a synthetic label matches what was actually generated. Slightly wrong synthetic boundaries collapsed a pancreatic tumor segmenter from 48.48% to 32.49% Dice (Optimizing Synthetic Data for Enhanced Pancreatic Tumor Segmentation). Inpainting models do not stay inside the mask they are given. They bleed texture past it or leave parts of it unchanged. Because each synthetic image is an edit of a known real image, we never have to trust the requested mask. The pixel difference between the healthy original and the edited result shows exactly what the generator changed, and that changed region becomes the label.

Core Claim

In the few-shot field regime (1 to 10 annotated images per disease), detectors trained with diffusion-inpainted lesions on real healthy field leaves will beat both real-only training and copy-paste of the real lesions. This holds when the box labels are taken from the generator's actual edit footprint rather than from the mask it was asked to fill. The footprint labeling is the specific new ingredient. The volume of synthetic data and the lesion size distribution are tuned against two known effects from medicine. First, a 1× synthetic ratio helped lung nodule detection while 3× hurt it (Synthesizing Diverse Lung Nodules Wherever Massively). Second, oversized synthetic tumors improved generalization (Optimizing Synthetic Data for Enhanced Pancreatic Tumor Segmentation).

Concretely, let $x$ be a healthy field image, $x'$ the inpainted result, and $G_\sigma$ a small Gaussian blur. The footprint mask is

$$ M(p) = \mathbb{1}\big[\, \| G_\sigma * (x' - x) \|(p) > \tau \,\big], $$

where $p$ indexes pixels and $\tau$ is a threshold calibrated on the real lesions. The edit footprint is then intersected with a dilated version of the requested mask, which discards global color drift far from the edit. Each connected component of the footprint becomes a box. If the component is too small, too fragmented, or spills onto non-leaf pixels (according to a leaf mask from the healthy image), the sample is rejected.

Position Relative to Prior Work

Prior agricultural detectors supply the target setting and the reference numbers but assume plentiful labels. TomFormer, a 1.30M-parameter CNN–Transformer detector deployed on a greenhouse robot, reaches 81% mAP on the in-field PlantDoc benchmark and 87% on KUTomaDATA, with ambiguous classes like early blight at 80% (Early and Accurate Detection of Tomato Leaf Diseases Using TomFormer). DHLC-DETR reaches 97.44% mAP on rice leaves from IDADP using 600 raw images per disease (Disease Detection and Identification of Rice Leaf Based on Improved Detection Transformer). That paper also reports confusion with dead grass and soil, which motivates our rejection of edits that land off the leaf. None of these works operates with a handful of labels.

The closest generative agricultural work places diffusion inside the detector. That model denoises feature maps and reaches 92% mAP@50 on five soybean diseases with 803 to 1,792 images per class (A Diffusion-Based Detection Model for Accurate Soybean Disease Identification). It produces no new training images and is untested at low shot counts. We do the opposite: diffusion as an external producer of labeled images. We also test whether the internal variant survives at k-shot.

Few-shot agricultural work relies on language rather than pixels. VLCD generates text descriptions with a vision-language model to adapt CLIP for 54-class classification (Few-Shot Image Classification of Crop Diseases Based on Vision-Language Models). AgEval shows GPT-4o identification F1 rising from 46.24% at 0-shot to 73.37% at 8-shot, with per-class coefficients of variation of 26–58% and a 15.38% jump when an exact-class example is supplied (AgEval). Both are image-level, and neither localizes lesions. The "exact-class example matters" finding motivates us: our generator manufactures many exact-class, in-field examples for the tail diseases.

The method itself is borrowed from medicine. Lesion insertion into healthy context is taken from MCGAN, which uses a separate discriminator for blending with surrounding tissue (Synthesizing Diverse Lung Nodules Wherever Massively), and from mask-guided latent diffusion (Optimizing Synthetic Data for Enhanced Pancreatic Tumor Segmentation). Those papers also constrain us, since they document the ratio cliff and the boundary sensitivity. Full-image text-to-image synthesis improved a skin classifier from 56.0% to 63.0% (Using Large Text To Image Models with Structured Prompts for Skin Disease Identification). However, it produces images without real backgrounds or localization labels, so it serves here as a competitor.

Research Program

We first build the generator. A pretrained text-to-image inpainting diffusion model is adapted per disease with a lightweight low-rank adapter trained only on the k annotated field images. Real lesion regions are masked out and reconstructed, with a short structured prompt naming the crop, disease, and stage, following the slot structure of Rajapaksa et al. Optionally, lab images of the same disease from PlantVillage are added for texture only. Their uniform backgrounds are masked away, so they cannot teach background cues.

Next comes placement. Healthy leaves in field images are segmented with a generic segmenter. Masks are sampled from the shapes of the k real lesions under random scaling, rotation, and elastic perturbation. They are placed only on leaf pixels, with counts per leaf matched to the real images.

Third, every edit is relabeled by the footprint rule above and filtered. Fourth, a standard lightweight detector is trained on real plus synthetic images. We sweep the synthetic-to-real ratio (0.5×, 1×, 3×, 10×) and the lesion size distribution: matched to real, shifted upward toward the largest quartile, or uniform.

The analysis asks where the gains come from. It breaks results down per class, per lesion size, and per false-positive type (dead tissue, soil, shadow). It also tracks the variance of results across classes and across random draws of the k shots, since AgEval's results suggest instability is the main practical problem.

Baselines

The naive reference point is a COCO-pretrained detector fine-tuned on only the k real annotated images per class with standard flips, crops, scaling, and color jitter. Every other method uses this detector, the same schedule, the same compute, and the same hyperparameter search (a fixed budget of trials on a held-out k-shot validation split drawn from training images, never the test set). Beating it means a gain in mAP@50 on the real field test set that exceeds the spread across five random draws of the k shots.

The strongest simple competitor is copy-paste augmentation. The k real lesions are cut out using their boxes refined by a generic segmenter, then pasted onto the same healthy field leaves with the same placement, ratio, and size policy, with Poisson blending at the seams. It isolates our key question, which is whether generated lesions add anything beyond relocated real ones. To beat it, the diffusion pipeline must outperform it at k=1 and k=5 on PlantDoc and KUTomaDATA, with the gain concentrated on field images whose lighting differs from the source shots.

The full-image text-to-image competitor follows Rajapaksa et al.: structured prompts generate whole diseased-plant images, and boxes come from the k-shot detector's own pseudo-labels. It is given the same image budget.

The architectural competitor is the internal-diffusion soybean detector of Yin et al., retrained under the identical k-shot splits on the soybean data and on PlantDoc. This tests whether feature-space denoising alone suffices when labels are scarce.

Reference ceilings come from published full-data results: TomFormer at 81% mAP on PlantDoc and 87% on KUTomaDATA, DHLC-DETR at 97.44% on IDADP, and Yin et al. at 92% mAP@50 on soybean. No reported result found on Lacuna for few-shot (1- to 10-shot) detection on PlantDoc, KUTomaDATA, or IDADP, so our naive and copy-paste baselines establish that regime.

Two ablations isolate the proposal's ingredients. The first keeps every generated image fixed and swaps footprint-derived labels for the requested-mask labels, which tests label fidelity. The second removes the leaf-only placement and off-leaf rejection, which tests the background-confusion fix.

Why not just…?

Why not just copy-paste the real lesions? This is the real threat. Copy-paste gives perfectly faithful lesion texture and exact labels for free. Its known weakness is "transition realism," the seam and illumination mismatch that the medical inpainting literature had to design around. MCGAN dedicates a whole context discriminator to it (Synthesizing Diverse Lung Nodules Wherever Massively). Copy-paste also cannot produce morphologies beyond the k shots. Even so, at k=1 the diffusion adapter has no more information than copy-paste does, and it could simply add artifacts. If copy-paste matches diffusion inpainting on the field test sets, we will report that cheaper method as the recommendation. The leaf-aware placement and ratio and size calibration remain the contribution. The comparison is decisive because both arms share placement, ratio, and detector.

Why not just use a few-shot vision-language model? AgEval shows 8-shot GPT-4o reaching 73.37% F1 for identification, but with 26–58% per-class variation. It works on whole images through remote APIs, which conflicts with on-robot deployment, and it gives no lesion locations for counting or severity. VLCD likewise only classifies. We still run GPT-4o-style in-context classification on image crops as an image-level reference, so that readers can see whether localization costs accuracy.

Why not just fine-tune a pretrained detector with heavy standard augmentation? That is our naive baseline. Transfer learning works well on clean lab images: VGG16 reaches 90.4% on PlantVillage apple severity (Automatic Image-Based Plant Disease Severity Estimation Using Deep Learning). Geometric augmentation, however, never places a lesion in a new field context, which is what the lab-to-field gap requires. If this baseline is within noise of our method at k=10, we will say the method matters only at the lowest shot counts.

Proposed Experiments

The main benchmarks are PlantDoc and KUTomaDATA, because both contain field or greenhouse imagery and healthy-leaf classes that supply canvases. IDADP rice and the soybean dataset of Yin et al. serve as secondary checks. For each, we build k-shot training splits with k in {1, 5, 10} annotated images per disease, using five random draws. Healthy training images are used without lesion labels. Test sets remain the original real annotated images. The metrics are mAP@50 and mAP@50:95, per-class average precision, the coefficient of variation of average precision across classes and across draws, and false-positive rates on dead tissue, soil, and shadow regions. The latter are tallied by a manual audit of 200 false positives per method.

Three checks decide the claims. First, at a fixed 1× ratio, footprint labels must beat requested-mask labels on identical images. If they do not, the label-fidelity argument fails, and generator bleed is not a real problem in this domain. Second, diffusion inpainting must beat copy-paste on field test images, with the margin growing as source and target lighting diverge. Lighting difference will be binned by the luminance difference between source lesion and target leaf. If neither condition holds, the generator adds nothing. Third, the ratio sweep is expected to show a peak followed by a decline, as Han et al. found. We test whether footprint filtering moves that peak to a higher ratio, which would indicate that the decline came from label noise rather than from generator style. A secondary check tests whether the "large lesion advantage" from pancreatic tumors carries over to small early-stage foliar spots. That would matter practically, since early detection is the point of scouting.