Peer-Reviewing the Annual Report: Verified Digital Maturity and the Subsidy Turning Point

Much of the evidence on when digital investment pays off, and when government subsidies for it stop working, rests on one measurement habit. Researchers count digital-technology keywords in firms' annual reports and treat the count as the firm's "digital maturity." A firm's disclosures, though, are written partly to win subsidies. If subsidized firms learn to talk digital without building digital, then the U-shaped curves and turning points reported in this literature may partly be curves in rhetoric rather than in capability. We propose to measure digital maturity the way a journal decides whether a paper's claims hold up. A structured panel of language-model roles extracts each digital claim a firm makes, challenges it, gathers evidence from the firm's own filings, and rules on whether the claim is substantiated. We then re-run the central results of the subsidy and maturity-threshold literature with this verified measure.

The payoff is a direct test of an open question. Digitization and Green Technology Innovation of Chinese Firms Under Government Subsidy Policies finds that subsidies raise green innovation up to about 1,076,000 CNY and reduce it beyond that point. The authors cannot say whether the decline reflects rent-seeking, superficial patenting, or managerial exhaustion. If the gap between what a firm claims and what can be verified widens with subsidy size, and widens most past the turning point, the rent-seeking reading gains direct support for the first time. If the gap stays flat, that reading loses its most natural mechanism.

Core Claim

We make two claims. The first is a measurement claim: a keyword-based digital index mixes real capability with strategic signaling, and a claim-by-claim verified index separates the two. The second is substantive and follows from the first: the difference between the two indices, which we call claim inflation, is not random noise. It rises with subsidy receipt. Because it is systematic, firm fixed effects cannot remove it, and it biases the estimated maturity thresholds and subsidy turning points.

Let $K_{it}$ be the standard keyword index for firm $i$ in year $t$, and $V_{it}$ the verified index built from the same report, both standardized. Claim inflation is $G_{it} = K_{it} - V_{it}$. The key test estimates

$$G_{it} = \alpha_1 S_{it} + \alpha_2 S_{it}^2 + \gamma' X_{it} + \mu_i + \lambda_t + \varepsilon_{it},$$

where $S_{it}$ is government subsidy received, $X_{it}$ are firm controls, and $\mu_i$ and $\lambda_t$ are firm and year fixed effects. Rent-seeking predicts $\alpha_1 > 0$, with inflation growing fastest near and above the turning point from Zhang et al.

The "editorial board" in this proposal is a method for producing $V_{it}$, not a metaphor for corporate governance. It is justified only if it beats simpler scorers, and we test that directly.

What prior work establishes and leaves open

Three lines of work motivate the problem. First, threshold studies: Does Artificial Intelligence Promote Sustainable Growth of Exporting Firms? finds a U-shaped effect of AI on export-firm growth with a turning point at an AI index of 2.38. DIGITAL TECHNOLOGY, ENVIRONMENTAL QUALITY AND GREEN FINANCE finds a U-shape at the city level. The Non-Linear Impact of Digitalization on the Performance of SMEs finds the opposite curvature, an inverted U, in a 352-firm survey. The firm-level studies build maturity from annual-report text, and the research file flags this explicitly as capturing "public relations signaling" rather than operational deployment. We do not dispute that curvature exists. We ask where the turning points fall once signaling is removed.

Second, subsidy studies. Zhang et al. supply the inverted-U subsidy curve and the rent-seeking puzzle. The Impact of Corporate Digital Transformation on Green Technological Innovation in the Context of "Dual-Carbon" Goals reports a small but significant direct effect across 24,643 firm-years and finds that returns decay over time. The low-carbon pilot city evaluation by Liu et al. finds policy effects only among high-maturity firms. All three condition on a text-based maturity measure. If that measure is inflated by subsidy-seeking, the "high-maturity firms respond" result could partly reflect "firms that talk well respond," and the observed decay could reflect inflation saturating.

Third, peer review and multi-agent work. We borrow methods from these papers; none of them supplies evidence on firms. PeerRead shows that acceptance correlates with surface features such as appendices, equation density, and benchmark terminology. That is the review-world analogue of keyword counting, and it motivates evaluating claims rather than vocabulary. FIRSTPASS shows that the quality signal sits in multi-round exchanges between reviewer and author, and that a fine-tuned model predicts round outcomes well. That motivates a challenge-and-response structure over one-pass scoring. Embodied LLM Agents Learn to Cooperate in Organized Teams shows that a designated leader improves a three-agent team's completion time by 9.76% in a household simulator. That paper also constrains us: its evidence comes from closed, simple tasks, so we treat any benefit of hierarchy in a high-stakes evaluation task as unproven until our ablations show it. ResearchTown simulates review writing, but its reviews reach only 0.49 semantic similarity to human reviews. That is a reminder that generating plausible review text differs from making correct judgments.

No work in the file verifies firms' digital disclosures claim by claim, and none tests whether subsidy receipt predicts the gap between claims and evidence.

Research Program

The verification pipeline. The pipeline runs in three roles over one firm-year.

A reviewer role reads the management discussion section of the annual report. It extracts atomic digital claims, for example "deployed machine-vision inspection on two production lines" or "established a data middle platform." It tags each claim by type: infrastructure, process deployment, personnel, or product.

A respondent role then searches the firm's evidence dated on or before year $t$ for support. The evidence sources are:

  • financial-statement notes, such as software and intangible asset lines, capitalized development, and IT-related capital spending;
  • disclosed headcount by function;
  • prior-year reports, to check whether a claimed deployment was earlier described as planned.

An editor role adjudicates each claim as substantiated, partially substantiated, or unsupported, and must cite the evidence it relied on. $V_{it}$ is the count of substantiated claims, weighted by claim type, and standardized like $K_{it}$.

Two design rules protect the downstream econometrics. First, evidence is restricted to information available at or before $t$, so the measure cannot use future outcomes. Second, digital-economy patent counts are withheld from the respondent entirely and reserved for validation.

Gold labels. Annotators with accounting training will label a stratified sample of several hundred firm-years at the claim level. Strata cover subsidy level, industry, and ownership. Annotators see the same evidence the pipeline sees. Two annotators label each claim, and their agreement sets the ceiling against which the pipeline is judged.

Replication with the swapped measure. We use the panel structure of Zhang et al.: Shanghai and Shenzhen A-share firms, 2015–2022, with the CSMAR financial and subsidy fields they draw on. On this panel we re-estimate three results with $V$ in place of $K$:

  • the subsidy inverted U, with digital transformation as mediator;
  • the heterogeneous pilot-policy effect by maturity, following Liu et al.;
  • the AI U-shape and its turning point, following the export-firm study on its 15,023-observation panel.

We report how turning points move, with bootstrap intervals clustered by firm. We also estimate the claim-inflation equation above, controlling for financing constraints in the spirit of The impact of digital intelligence technologies on innovation performance. This keeps the test from simply picking up that constrained firms both chase subsidies and write aspirational reports.

A secondary check on the architecture. Before trusting the challenge-and-response structure on firms, we test it where ground truth exists. On FIRSTPASS, we predict standard versus extended review from the manuscript and first-round review. We compare the three-role pipeline against a flat single-call scorer on the same backbone, with a matched token budget. This check asks only whether structure helps at fixed compute. We do not expect a prompted pipeline to beat a fine-tuned model.

Baselines

The naive reference. The naive reference is the keyword index $K_{it}$ as constructed in the digital-transformation literature used by Zhang et al. and Gan et al. On gold labels it is judged by how well it ranks firms by substantiated capability. In the econometrics it is the incumbent measure whose turning points we re-estimate. Beating it means higher rank correlation with gold-labeled substantiated claims, and higher correlation with the withheld patent counts, at the firm-year level.

The strongest competitor. The strongest competitor is a single-pass language-model scorer. It uses the same backbone, reads the report plus exactly the same retrieved evidence bundle our respondent sees, and outputs a verified-claim score in one call. It is run with the same total token budget as the three-role pipeline and gets the same number of prompt-development iterations on a held-out development split. It is the real threat because it has access to all the same information and differs only in structure. The hierarchy is justified only if it beats this scorer on claim-level macro-F1 against gold labels by more than the bootstrap interval. It must also yield a claim-inflation estimate that is not sensitive to the swap.

No reported result was found on Lacuna for any automated verification of corporate digital disclosures, so there is no published number for this task.

Reference points for the secondary FIRSTPASS check. FIRSTPASS reports 80.5% accuracy and 78.2% macro-F1 for fine-tuned Qwen2.5-7B-Instruct, and a zero-shot Gemini-3.1-flash-lite-preview 10.4 points lower. We report both as references. Our pipeline will be compared primarily to its own flat single-call counterpart. We also run it on the 7B backbone, fine-tuned and not, so that the architecture's contribution can be read at fixed model and fixed training.

Ablations. Two ablations isolate the key ingredient.

  • The first removes the respondent's evidence retrieval, so the editor judges claims from the report text alone. This separates the value of cross-document evidence from the value of role structure.
  • The second removes the editor, so claims count as substantiated if the respondent finds any match. This tests whether adjudication adds anything over retrieval.

Why not just…?

Why not just use a single strong language model with the same evidence? This is the most serious alternative, and we may lose to it. Nothing in the file shows that role hierarchy helps on evaluation tasks. The 9.76% gain in Embodied LLM Agents comes from coordinating household chores. The matched-budget comparison in the Baselines settles this. If the single-pass scorer ties the pipeline, we adopt it as the measurement tool and drop the architectural claim. The substantive contribution survives unchanged, because it depends on having a verified measure, not on how the measure was produced.

Why not just use hard proxies such as digital patents or IT asset lines? Patents are themselves a subsidy target: Zhang et al. name strategic filing as their main limitation. Many firms that deploy software never patent it. Asset lines are coarse and inconsistently itemized across firms. We keep patents out of the pipeline and run a horse race. We compare the keyword index, an asset-line index, and $V$ on how well each predicts gold-labeled capability. If the asset-line index alone matches $V$, the cheaper proxy wins and we say so.

Why not just rely on firm fixed effects to absorb measurement noise? Fixed effects remove signaling that is constant within a firm. They do not remove signaling that moves with subsidies over time, and that is exactly the pattern the rent-seeking hypothesis predicts. The claim-inflation regression tests whether that time-varying component exists. If $\alpha_1$ is indistinguishable from zero and the turning points do not move, the existing literature's measurement choice is vindicated. That result is worth publishing too.

Why not just attribute the post-turning-point decline to liquidity or capital constraints? A liquidity story predicts that innovation keeps rising with subsidy. Zhang et al. observe the opposite, so pure liquidity cannot explain the decline. More to the point, a liquidity story makes no prediction about claim inflation at all. We include financing-constraint controls so that any inflation–subsidy association cannot simply reflect financially stressed firms writing optimistic reports.

Decisive Checks

The first check is measurement validity. On the gold-labeled sample, $V$ must beat $K$ in rank correlation with substantiated capability. Out of sample, $V$ must correlate more strongly than $K$ with the withheld digital-economy patent counts. If $V$ fails either test, the project stops at a negative result about automated verification.

The second check is the architecture comparison against the matched single-pass scorer and the two ablations. It determines which pipeline produces $V$.

The third and central check is substantive. We estimate $\alpha_1$ and $\alpha_2$ in the claim-inflation equation, then re-estimate the subsidy turning point with $V$ in place of $K$. The rent-seeking reading of the Zhang et al. decline is supported if two things hold together. First, inflation rises with subsidy, significantly and robustly to financing controls. Second, the verified-measure turning point shifts, or the decline flattens, once inflation is removed. It is undercut if inflation is flat in subsidy.

We will also report whether the heterogeneous pilot-policy effect of Liu et al. and the export-firm AI turning point survive the swap. All results come from listed Chinese firms. The survivorship bias the file notes, which leaves out small unlisted firms stuck in the early friction phase, limits how far any conclusion extends.