Thousands of plant, microbial and animal stress papers each year report the same small panel of measurements. They measure the activities of superoxide dismutase (SOD, the enzyme that turns superoxide into hydrogen peroxide), catalase (CAT) and peroxidases (POD, APX, GPX), which then remove that peroxide. They also report malondialdehyde (MDA, a byproduct of membrane damage) and electrolyte leakage. Almost every abstract in this literature concludes that a protective treatment "enhanced the antioxidant system." Keyword co-occurrence analysis maps a field by counting which terms appear together in papers. Here it would faithfully recover that SOD, CAT, POD and MDA always travel together. It cannot tell us whether the measurements behind those words actually moved together, in the stated direction, by a meaningful amount.
This note proposes building two graphs over the same nodes from the same corpus and measuring where they disagree. The first is the usual word graph, taken from abstracts. The second is a measurement graph, in which two assays are linked when their extracted effects change together across experiments, with sign and size taken from results tables. Disagreement between the graphs has a concrete meaning. It marks places where the field's narrative has drifted from its data, for example abstracts that claim a coordinated enzyme increase while the tables show catalase unchanged. The claim is that this abstract-versus-data discordance can be measured at scale, that it is structured rather than random noise, and that it is the missing quantity that makes co-occurrence maps of experimental fields interpretable.
Why this and not the obvious idea
The obvious intersection is a "signed co-occurrence graph." One would extract directed, signed relations of the form "compound X increases SOD" and project them into a network, so that ethephon raising SOD in freezing tea shoots (Chen et al.) and curcumin photodynamic treatment depleting SOD, CAT and GPX in bacteria (Li et al.) no longer merge into one positive edge. That idea is largely already available. RegulaTome built a corpus of typed, directed, signed relations between proteins and chemicals, trained an extractor, and ran it over PubMed and PMC to produce more than 18 million relations. Filtering those relations to antioxidant enzymes and drawing a graph would be an afternoon's work. It would still inherit the core problem: its signs come from the prose of the paper, which is exactly the narrative whose faithfulness is in question. RegulaTome also records no magnitudes, no null results and no experimental context.
The proposal therefore changes the object of study. It does not ask "what does the literature say about antioxidant enzymes." It asks "how well does what the literature says match what it measured." That requires a second, independent source of signal, the numbers in results tables and figures. It also requires a unit-free way to compare those numbers across papers.
Core Claim
Within a single experiment, the treated-to-control ratio of an enzyme activity is unitless even when papers use incompatible activity units, such as U/mg protein versus percent inhibition. This largely sidesteps the assay-standardization problem that makes cross-study enzyme comparison hard to automate. For experiment $k$ and assay $a$ we use the log response ratio
$$ r_{k,a} = \ln \frac{\bar{x}^{\text{treated}}_{k,a}}{\bar{x}^{\text{control}}_{k,a}}, $$
where $\bar{x}$ is the reported mean activity or content. The measurement graph links assays $a$ and $b$ with weight equal to the correlation of $r_{k,a}$ and $r_{k,b}$ across all experiments $k$ that report both. The word graph links them with the usual normalized co-occurrence count from abstracts.
The central quantity is per-paper discordance. It is the fraction of assays whose abstract-stated direction differs from the table-derived direction, or which the abstract calls changed while the table shows a non-significant difference. We predict three things. Discordance is non-trivial. It concentrates in specific assays, for instance the peroxidases more than SOD, and in specific study designs, for instance detached-organ systems like the tomato-leaf model in Zhang et al., which reports enzyme changes only qualitatively. Finally, the measurement graph shows weaker CAT–POD coupling than the word graph implies. The cascade logic, in which SOD produces peroxide for CAT and POD to consume, predicts that the two peroxide-removing branches can compensate for each other rather than always rising together.
Research Program
The corpus is PubMed Central open-access papers on stress physiology and toxicology that report at least one of SOD, CAT, POD/APX/GPX or MDA, found by keyword query. Full text is required because the second graph lives in tables. The corpus spans the application regimes in the file: abiotic stress such as freezing (Chen et al.) and lead (the Cassia tora co-expression study), senescence and postharvest work (melatonin in blueberries), pathogen and biocontrol studies (Bacillus amyloliquefaciens against bacterial wilt; Fusarium in faba bean), and animal toxicology (abamectin in zebrafish).
Three components are built. The first is a standard co-word pipeline in the style of the bibliometric studies in the file: author keywords plus abstract noun phrases, a symmetric count matrix, and Leiden clustering. We treat this as the reference representation of "what the field says."
The second is an experiment-level extractor. For each treatment–control contrast it records the stressor, treatment, organism, tissue and time point. For each assay it records the abstract-stated direction and, from the tables, the means, the dispersion where available, and the stated significance.
The third is the pair of graphs, a discordance score per paper and per assay, and a regression of discordance on design covariates: organism class, detached versus intact tissue, whether the assay was the paper's headline outcome, and journal.
The extractor is the main engineering risk. PubMedQA found that about 21% of abstract contexts contain raw statistics with no verbal description. Its BioBERT model reached 68.1% accuracy, against 78.0% for humans and 55.2% for guessing the majority class. Reading numbers out of biomedical text is therefore not solved, and the extractor must be validated against human annotation rather than trusted.
Baselines
The naive reference point is the "canonical story" rule. It predicts that every protective treatment raises SOD, CAT and POD and lowers MDA, and that every damaging treatment does the reverse, using only the paper's framing of the treatment. If this rule already matches table-derived directions almost perfectly, the literature is faithful and our discordance measure will read near zero. We will report its direction accuracy against the hand-annotated table gold set. That result is itself informative.
The standard bibliometric baseline is an undirected keyword map built with the tooling used by Zou et al. and Liu et al.: VOSviewer and CiteSpace-style counts with Leiden clusters. It runs on the identical corpus. We compare its edge weights with the measurement graph by rank correlation. "Beating" it here does not mean higher accuracy. It means showing that the two graphs disagree in a statistically detectable, assay-specific way that the word graph cannot see.
The strongest existing method for signed relations is the RegulaTome extractor. It reports 66.6% F1 over typed, directed, signed relations on its own corpus, against a 91% F1 inter-annotator ceiling. We apply its model, or its released extractions filtered to our papers and enzyme entities, to the same abstracts. We score its enzyme-level signs against our table-derived gold labels. For the specific task of assay-level effect extraction from antioxidant enzyme papers, no reported result was found on Lacuna, so we create a gold set. Two annotators will label roughly 300 papers from tables, and we report their agreement. All systems are scored on held-out papers only.
The large-language-model baseline gives a prompted model abstracts alone, with the same token budget and a fixed prompt tuned on the same development papers as our extractor.
The key ablation removes table access from our own pipeline. If measurement-graph edges built from abstract-only extraction match those built from tables, then the table step, which is the proposal's distinguishing ingredient, adds nothing.
Our method beats the field if two conditions hold. Table-derived directions must disagree with both the canonical-story rule and the RegulaTome signs on a measurable share of assays, beyond the disagreement between our two human annotators. The disagreements must also be predicted by design covariates rather than scattered at random.
Why not just…?
Why not just run VOSviewer or CiteSpace on an "antioxidant enzyme" search? It will return a dense cluster of SOD, CAT, POD, MDA and reactive oxygen species. That is true, and it tells us only that these terms are reported together, which everyone already knows. The file's bibliometric studies (Guo et al.; Zou et al.) produce descriptive clusters that do not engage with mechanism or data. We keep this map as one of our two graphs precisely because the comparison needs it.
Why not just project RegulaTome's 18 million signed relations onto a graph? This is the real threat for the sign question. If RegulaTome's abstract-derived signs agree with table-derived signs at roughly human-agreement level, our contribution shrinks to magnitudes and null results. The baseline experiment above settles this directly. A priori, the approach cannot detect discordance, because it reads the same narrative text whose accuracy we are testing. It also has no representation of "not significantly changed."
Why not just do a conventional meta-analysis with language-model extraction? A meta-analysis pools effect sizes for one question, such as the effect of melatonin on CAT. It does not ask whether the field's text faithfully represents its data. It also would not compare the co-occurrence structure of words with the co-response structure of measurements. Our extractor effectively produces meta-analysis-ready data as a by-product. If reviewers only want pooled effects, the table component alone suffices, and we say so. The additional scientific claim lives in the graph comparison. Retrieval-and-summarize pipelines are also known to produce fluent but weakly grounded syntheses: SurGE reports end-to-end citation recall often below 10%, and SurveyG needed explicit graph structure to improve on this. This is why we score extraction against tables rather than against generated summaries.
Proposed Experiments and Decisive Checks
The first experiment validates extraction. On the gold set we report assay-level direction accuracy and log-ratio error for our extractor, the language-model baseline and RegulaTome. Worked examples are chosen for their contrasting reporting styles. Chen et al. give explicit percentages: SOD about 39.1%, POD about 33.0%, CAT about 34.9%. Zhang et al. report direction only. Li et al. report depletion.
The second experiment estimates discordance rates per assay and per design class, with bootstrap intervals over papers. We compare these rates with the disagreement between the two annotators, which serves as a noise floor.
The third compares the graphs using rank correlation of edge weights between the word graph and the measurement graph, and tests the specific CAT–POD coupling prediction.
The fourth checks biochemical coherence. It measures how often a paper reports SOD up, both peroxide-removing branches flat or down, and peroxide down. This pattern is not impossible, but it is worth flagging, and we test whether its rate tracks discordance.
The decisive negative outcome is that discordance does not exceed annotator noise and the two graphs correlate near perfectly. That would mean abstracts are a faithful summary of this literature, and cheap text-only co-occurrence is adequate for experimental fields of this kind. That is a useful result for bibliometrics.
The decisive positive outcome is structured discordance concentrated in identifiable assays or designs. That would give science-mapping a validity check it currently lacks, and would give experimentalists a concrete list of places where the antioxidant-enzyme narrative outruns its measurements.
References
- Exogenous Activation of the Ethylene Signaling Pathway Enhances the Freezing Tolerance of Young Tea Shoots by Regulating the Plant’s Antioxidant System
- Inactivation of Bacillus subtilis by Curcumin-Mediated Photodynamic Technology through Inducing Oxidative Stress Response
- RegulaTome: a corpus of typed, directed, and signed relations between biomedical entities in the scientific literature
- Exogenous Melatonin Delays Methyl Jasmonate-Triggered Senescence in Tomato Leaves
- Correction: Co-expression of Cassia tora 1-deoxy-D-xylulose-5-phosphate synthase and 1-deoxy-D-xylulose-5-phosphate reductoisomerase enhances tolerance of transgenic Nicotiana benthamiana to lead (Pb) stress
- Exogenous Melatonin Application Delays Senescence and Improves Postharvest Antioxidant Capacity in Blueberries
- Analysis of the Control Effect of Bacillus amyloliquefaciens C4 Wettable Powder on Potato Bacterial Wilt Caused by Ralstonia solanacearum
- Integrated metabolomics and metagenomics uncover pathogenic mechanisms of Fusarium wilt and faba bean defense responses
- Abamectin Causes Neurotoxicity in Zebrafish Embryos
- PubMedQA: A Dataset for Biomedical Research Question Answering
- A bibliometric analysis of the immune system and cognitive impairment: trends from 1985 to 2024
- Bibliometric analysis of the association between drinking water pollution and bladder cancer
- Application of Blockchain Technology in Environmental Health: Literature Review and Prospect of Visualization Based on CiteSpace
- SurGE: A Benchmark and Evaluation Framework for Scientific Survey Generation
- A Multi-Agent LLM Framework with Hierarchical Citation Graph for Automated Survey Generation