Registry-Resolved Citations for Financial Fact-Checking

Financial claims increasingly lean on science. A biotech press release cites a trial paper, a chipmaker's announcement points to a conference publication, and a social-media post about a company's breakthrough links a preprint. Language models asked to check these claims tend to treat the citation as a sign of authority rather than as something to check. Across four domains, appending any citation, real or invented, raises hallucination rates, by up to 22 percentage points for some models (AuthorityBench). In financial misinformation detection specifically, giving a fine-tuned model an extra metadata field made it worse, because it learned to key on whether the field was filled rather than on what it said (Team FMD LLM).

This proposal adds a step that comes from bibliographic record linkage. Before the fact-checker sees a citation, a separate resolver looks the citation up in a curated bibliographic index. It then replaces the raw citation string with what the lookup found. The result is either the verified paper's abstract, a typed report of which fields disagree (wrong author, year, or venue), or a flag that the paper could not be found. The fact-checker never reads author names, venues, or the bare fact that a citation is present. We expect this to remove the authority effect while keeping the real evidential value of the cited work. It should also produce a citation audit as a by-product, and that audit has value of its own. The audit has to tell apart two cases that look alike on the surface: a fabricated reference, and a real reference with garbled metadata. Medical retrieval studies find the second case to be the common failure of modern chatbots (Do AI chatbots find what experts would?).

Core Claim

Citation authority bias in financial fact-checking can be removed by changing what the model is shown, not what it believes. If citations are resolved against a registry before reasoning, two things should follow. First, the gap in veracity accuracy between "with citation" and "without citation" conditions should fall to near zero. Second, on claims where cited evidence actually bears on truth, accuracy should exceed simply deleting the citation. The second condition is the one that makes the idea worth doing, because deleting citations is the obvious cheap fix.

Two existing lines of work meet here, but neither does this. AuthorityBench measures susceptibility with a real-versus-fabricated design in general, science, law, and medical domains. It proposes no remedy, has no financial claims, and has no "real but garbled" condition. The resolver relies on Sefid et al.. Their Integrated Matching Model links noisy scholarly records to curated databases with F1 0.992. It does this by matching headers, gating out corrupted titles, and falling back to citation-list fingerprints. That work was built to reconcile catalogs and has never been placed in front of a reasoning model as an input filter. The section 2 overlap papers (Fin-Fact, Team FMD LLM) treat metadata as one more text field.

Research Program

The first piece is a benchmark. We start from the 3,369 expert-annotated claims in Fin-Fact, which pair text with charts and filing screenshots, and keep its multimodal inputs. Each claim is crossed with four citation conditions:

  1. No citation.
  2. Correct citation: a real, topically relevant paper with correct metadata.
  3. Garbled citation: the same real paper with perturbed metadata, such as swapped or misspelled authors, an off-by-one year, a wrong venue, or a truncated affiliation. These perturbations copy the error types documented in published correction notices such as Author Correction: Higher-order topology in bismuth and Correction to: Internal and external memory set containment join.
  4. Fabricated citation: a plausible but non-existent reference.

As in AuthorityBench, claim truth and citation condition are fully crossed, so citation status carries no label information by design. Real papers are drawn from the reference corpora used by Sefid et al., namely DBLP and Medline, so the resolver's index matches published linkage ground truth.

Balanced data cannot test the "citation evidence helps" half of the claim. For that we will also hand-annotate a smaller natural set of financial claims that already cite scientific work, drawn from biotech and semiconductor press releases and posts. In this set, whether the cited paper supports the claim is a real signal.

The second piece is the resolver. It parses the citation string, retrieves candidates lexically, and scores them with an IMM-style header matcher. A title-quality gate sends doubtful cases to fallback matching. The output is one of three typed results:

  • Verified: returned with the abstract.
  • Verified with discrepancies: returned with a list of the fields that disagree.
  • Unresolved.

The third piece is the fact-checker. We use the same model families as the baselines: Llama-3.2-1B fine-tuned in the prediction-first format from Kawamura, plus one frontier vision-language model used zero-shot. The fact-checker sees the claim, the image, and the resolver output. It never sees the raw citation.

Baselines

The naive reference point is the raw-citation condition. Each model receives the claim with the citation string pasted in, exactly as a deployed system would. Its "lift" defines the susceptibility we are trying to remove. Lift is the change in false-acceptance rate on false claims, and in rejection of true claims, relative to the no-citation condition.

The second reference is stripping: delete every citation and check the bare claim. On Fin-Fact without metadata, the strongest reported fine-tuned result is Micro-F1 0.7357 for prediction-first Llama-3.2-1B. Adding the sci_digest field drops this to 0.6969 (Team FMD LLM). The best reported text-only figure on Fin-Fact is GPT-4 at F1 0.76, and the best multimodal accuracy is Gemini Vision Pro at 0.58 (Fin-Fact). No reported result was found on Lacuna for financial claims with fabricated or garbled scientific citations, so the citation-condition numbers will be new.

The strongest competitor is a frontier model with live web search, instructed to verify the citation and then judge the claim. This is the most plausible way to do in one step what we do in two. Its known weaknesses come from Liu et al.: 39.2% average recall of expert-curated studies, a strong bias toward large landmark trials, and frequent wrong authors, years, and journals on real studies.

We also include a counterfactual-augmentation fine-tune. It uses the same Llama model trained on data where citations are randomly added, garbled, or fabricated independently of the label. This attacks the shortcut directly without any registry.

To keep the comparison fair, all fine-tuned variants share data splits, base model, and a fixed tuning budget of the same number of hyperparameter trials. All zero-shot variants share prompts except for the citation block.

We count the proposal as beating the field if it meets all of the following:

  1. Its lift is below 1 percentage point, while the raw-citation baseline's lift falls in AuthorityBench's reported +3 to +22 point range.
  2. On the balanced benchmark, its veracity F1 is within noise of stripping.
  3. On the natural set, it exceeds both stripping and the web-search verifier.

Ablations isolate the key ingredient. We will show the fact-checker the resolver's status alone, without the abstract, and the abstract alone, without the status. We will also swap the IMM-style resolver for lexical-only matching.

Why not just…?

Why not just strip citations from the input? On the balanced benchmark, stripping should be optimal, since citation status is independent of truth by construction. We expect it to tie our method there. That is why the natural set exists: the proposal stands or falls on whether resolved abstracts add accuracy where cited evidence matters. Stripping also cannot report that a press release cites a paper that does not exist, which is useful to analysts on its own terms. If the natural-set gain is not significant, the honest conclusion is that stripping is the fix and the resolver's value is only the audit.

Why not just tell the model to be skeptical of citations, or let it search? AuthorityBench found that venue prestige and author demographics barely mattered, while mere citation presence did. This suggests the bias acts at a level that instructions may not reach. The web-search baseline tests the stronger form of this alternative directly. Liu et al.'s finding that search-equipped models garble metadata on real studies predicts a specific failure: confusing garbled-but-real citations with fabricated ones. Our garbled condition is built to expose that failure.

Why not just fine-tune with randomized citations? This is the real threat. It may remove the shortcut that Kawamura observed without any external index. But the model still reads author and venue strings, so it may only learn to ignore citations, which amounts to learned stripping. It also cannot tell fabricated references from real ones. The comparison on the natural set, plus a citation-audit score that this baseline can only reach by guessing, settles whether the registry earns its cost.

Proposed Experiments and Decisive Checks

The primary experiment reports three numbers for every system under all four citation conditions on Fin-Fact-derived claims: veracity Micro-F1, susceptibility lift relative to no-citation, and citation-audit macro-F1 over the classes correct, garbled, and fabricated. The resolver is first validated against the manually labeled CiteSeerX–DBLP linkage sets from Sefid et al.. If its precision falls well below their reported 0.984, every downstream result inherits that error, and we report the error propagation explicitly.

The decisive checks are three:

  1. Does the lift vanish? The proposal must cut lift to near zero. A reduction that only matches counterfactual fine-tuning is not enough.
  2. Do resolved abstracts beat stripping on the natural set? This decides whether the idea is more than a filter.
  3. Does garbled get mistaken for fabricated? We measure how often each system labels a garbled real citation as fabricated. This is where record linkage should clearly beat an LLM with search, and where a failure would show the scholarly-integrity half of the idea adds nothing.

A deliberate limit: we do not add market price reactions as evidence. The returns around a claim's release date mix many confounds. Testing whether they help belongs to a follow-up study once citation handling is fixed.