Plant-disease recognition is a field where models routinely report near-perfect accuracy on laboratory photographs of single leaves on plain backgrounds, then fail on photographs taken in real fields. This proposal asks a question that neither the robustness literature nor the bibliometric literature has answered. Bibliometrics is the quantitative study of publication and citation records. The question is whether the methods the research community actually adopts and cites are the ones that hold up in the field. We will use citation data to pick the methods the field relies on. We will retrain all of them under one protocol, test them on in-the-wild datasets, and measure how well their laboratory ranking and their citation impact predict their field ranking.
The stakes are practical. A practitioner choosing a model for a farm robot or a phone app usually starts from the most-cited, highest-accuracy papers. If laboratory accuracy and citation impact are only weakly related to field performance, the published record is steering people toward the wrong models. That would justify concrete changes: a field-evidence index attached to published claims, and venue norms requiring a field test. If the relationship is strong, the "lab-to-field gap" is a uniform tax that does not distort choices between methods, and that is also worth knowing.
Core Claim
Our hypothesis has three parts. Among widely adopted plant-disease architectures, rank agreement between laboratory accuracy and field accuracy is modest. Citation impact is better explained by laboratory accuracy than by field accuracy. And a simple bibliometric feature predicts field transfer beyond what laboratory accuracy alone predicts. That feature is evaluation lineage: whether a paper, and the citation cluster it belongs to, ever evaluated on field data.
What is new is not cross-dataset testing, which is standard, and not bibliometric mapping, which is common. The new part is joining the two at the level of individual methods. The citation record defines the sample and serves as the outcome to be explained, and controlled field evaluation supplies the ground truth.
Grounding in Prior Work
The motivating evidence comes from robustness papers that document large drops between curated and wild data. WildDash built a driving benchmark annotated with ten hand-engineered hazard types such as glare and motion blur. It reports that a DRN-D38 segmentation model fell from about 64 mIoU on Cityscapes to 16.5 mIoU on WildDash, while the same model trained on the more diverse Mapillary Vistas reached 30.7. PlantSeg released an in-the-wild plant-lesion segmentation dataset on which ConvNeXt-L reaches only 46.24% mIoU and MobileNetV3 only 10.22%.
Those results contrast sharply with laboratory claims on PlantVillage: 94.32% validation accuracy for MobileNetV2 and 99.84% for FoldNet. The FoldNet paper complicates the story, because it also reports 90.49% on the in-the-wild FGVC8 set. Some lab-designed methods therefore transfer reasonably well. This is why the question has to be about rankings across many methods rather than anecdotes. TomFormer similarly reports mAP across PlantVillage (83%), PlantDoc (81%) and the greenhouse set KUTomaDATA (87%). This again shows that the size of the drop varies by method, and we want to explain that variation.
The bibliometric work constrains the design rather than supplying a ready method. Studies such as the kidney-medicine computer-vision mapping and the autoimmune AI bibliometric study chart publication growth and keyword clusters, and the latter names insufficient validation as a barrier. Neither measures model behavior, and we treat both as examples of the descriptive trap we want to avoid. The LEGO Serious Play mapping shows how co-citation analysis can expose a field resting on a narrow foundation. We borrow that diagnostic idea and apply it to evaluation datasets instead of theories.
Two papers supply technique. CORWA annotates the function of citation contexts, which we need in order to tell "cited as a baseline on field data" apart from "cited in passing." LLM-Enabled Scientific Knowledge Diffusion Analysis uses language models to extract structured information from papers, which we adapt to pull out which datasets each paper trains and tests on. PubMed knowledge graph 2.0 links papers to clinical trials, and our paper-to-field-test link plays the same role for vision.
Research Program
The first stage builds the corpus and the evaluation graph. We will collect plant-disease deep-learning papers from Web of Science and Scopus, and add arXiv to address the single-database siloing that the MCDM bibliometric study acknowledges. From each paper's text, a language-model extractor records the datasets used for training and for testing, flagged as laboratory or field. Its output is checked by hand on a random stratified sample, and we report extraction precision and recall.
We then build an evaluation-coupling graph. This is ordinary bibliographic coupling, except that papers are linked by shared test datasets rather than shared references. From it we compute, for each co-citation cluster, the fraction of papers that ever report a field evaluation.
The second stage selects methods. We rank architectures by co-citation centrality and citation counts normalized for publication year. We take roughly the forty most-adopted methods with public code or reimplementable descriptions, which yields a sample defined by community uptake rather than by our own taste. For segmentation we take a smaller set evaluable on PlantSeg.
The third stage is controlled re-evaluation. Every method is trained on PlantVillage with the same augmentation, schedule and hyperparameter-search budget. Each is then tested on PlantDoc and FGVC8, restricted to species-disease pairs shared with PlantVillage and mapped by hand. As a separate condition, each method is also trained on field data and tested on held-out field data. This separates "the architecture is fragile" from "laboratory training data is the problem."
The fourth stage is analysis. We compute rank agreement between laboratory and field accuracy. We regress year-normalized citation impact on laboratory accuracy and field accuracy jointly. We then test whether evaluation-lineage features add predictive power for field accuracy beyond laboratory accuracy and model size.
As a replication domain, we repeat the selection and analysis for driving segmentation models trained on Cityscapes and tested on WildDash. The small size of the WildDash hidden set (156 cases) limits precision there, and we will report it as a secondary check.
Baselines
The do-nothing reference is reported laboratory accuracy as the predictor of field performance: rank methods by what their papers claim on PlantVillage and see how well that recovers the field ranking. A slightly stronger version uses laboratory accuracy re-measured under our common protocol. This removes differences in reporting conventions, such as the 80/20 split used for MobileNetV2, and is the fair form of the naive predictor. Any bibliometric feature has to beat this.
The strongest competitor is model capacity, meaning parameter count and backbone family. The gap between ConvNeXt-L at 46.24% mIoU and MobileNetV3 at 10.22% mIoU on PlantSeg suggests that capacity alone may explain much of the variation in field performance. The countervailing data point is FoldNet's 90.49% on FGVC8 with 516k parameters, which comes from a single paper. We will fit capacity-only, lab-accuracy-only, and combined predictors on identical splits, using leave-one-method-out cross-validation. Every predictor gets the same feature-selection budget, and the comparison uses held-out rank correlation.
The ablation that isolates our key ingredient is evaluation lineage. We add the paper-level and cluster-level field-evaluation features to the capacity-plus-lab-accuracy model, then remove them again. Beating the baseline means a held-out improvement in rank correlation whose bootstrap confidence interval excludes zero.
For the citation analysis, the baseline is a model of citation impact using publication year, venue and laboratory accuracy. Our claim succeeds only if adding field accuracy contributes little to explaining citations while laboratory accuracy contributes a lot.
Why not just…?
Why not just run a cross-dataset benchmark of popular models and skip the bibliometrics? This is the most serious alternative, and the measurement half of our study is exactly that benchmark. What it cannot do alone is answer whether the publication system rewards the wrong signal. That requires two things only the citation record provides: a sample defined by community uptake, and citation impact as the variable to explain. If evaluation-lineage features add nothing beyond laboratory accuracy and capacity in the ablation, we will say so. The contribution then narrows to the incentive finding and the benchmark.
Why not just evaluate on synthetic corruptions like added blur, noise and fog? These are cheap, but WildDash shows that real hazards occur in combination and include out-of-scope inputs, where its Cityscapes model scored only 7.2. PlantSeg ties failures to irregular, tiny lesions that synthetic perturbations do not create. We will still compute corruption scores as an additional predictor. If they predict field rank as well as real field tests do, that is a useful and cheaper result.
Why not just do a manual systematic review of which papers tested in the field? A careful review could extract the same evaluation labels for a few hundred papers, and our hand-checked sample is effectively such a review. The automated extractor matters only because the coupling graph needs the full corpus. If extraction quality is poor, we will fall back to manual coding of the selected methods and their citing clusters.
Proposed Experiments and Decisive Checks
The first decisive check is the rank agreement (Kendall's tau) between laboratory and field accuracy across the roughly forty retrained methods, on PlantDoc and FGVC8. If tau is high, for example above 0.7 with a confidence interval excluding moderate values, the central hypothesis is falsified. Laboratory rankings would then be a safe guide, and the note's recommendation reverses.
The second check is whether year-normalized citation impact is explained by laboratory accuracy and not by field accuracy, controlling for venue and year.
The third check is whether evaluation-lineage features improve held-out prediction of field rank over the capacity-plus-lab-accuracy baseline.
The fourth check uses the field-trained condition. If field-trained rankings agree with laboratory-trained field rankings, then architecture matters and data provenance does not. If they disagree, the training data, not the method, drives transfer.
For context on the strongest existing field numbers, the reference points we expect to reproduce approximately are ConvNeXt-L at 46.24% mIoU on PlantSeg, FoldNet at 90.49% accuracy on FGVC8, and TomFormer at 81% mAP on PlantDoc. Reproduction within a few points validates our protocol before any of the meta-scientific comparisons are interpreted.
References
- WildDash - Creating Hazard-Aware Benchmarks
- A Large-Scale In-the-wild Dataset for Plant Disease Segmentation
- Plant Leaf Disease Detection using MobileNetV2
- Lightweight Isotropic Convolutional Neural Network for Plant Disease Identification
- Early and Accurate Detection of Tomato Leaf Diseases Using TomFormer
- Kidney medicine meets computer vision: a bibliometric analysis
- Artificial intelligence in autoimmune diseases: a bibliometric exploration of the past two decades
- LEGO® Serious Play® Research: A Bibliometric Mapping of Themes, Trajectories and Frontiers (2015–2025)
- CORWA: A Citation-Oriented Related Work Annotation Dataset
- LLM-Enabled Scientific Knowledge Diffusion Analysis
- PubMed knowledge graph 2.0: Connecting papers, patents, and clinical trials in biomedical science
- A Bibliometric Analysis of Research on Multiple Criteria Decision Making