Planning Literature Surveys Around Statistically Surprising Citation Structure

Automated literature-survey writers now fetch hundreds of relevant papers and draft section by section. The weakest part of these systems is the plan: deciding which papers belong together and what story each section tells. Current planners group papers by topic similarity or anchor them on highly cited "core" papers. Neither choice captures what an expert survey actually narrates: lineages, where one idea spawns another and both feed a third; forks, where rival methods attack the same problem and disagree; and bridges, where one paper joins two communities. We propose building the plan from network motifs. A network motif is a small wiring pattern, such as a three-node triangle, that occurs in a network far more often than it would in randomized copies of that network with the same number of links per node. The idea is to detect which small patterns of typed citations ("extends", "uses", "compares against", "contradicts") are over-represented in a topic's citation graph. Each instance of an enriched pattern then becomes a planned narrative unit that a language model writes up.

In two sentences: first, label each citation among the retrieved papers with its function and find which small citation patterns occur more than chance predicts, where chance is a randomization that respects time order and citation counts. Second, turn instances of those patterns into the section and paragraph structure of the survey, so the writer is told which papers to relate and how. The claim that separates this from "give the LLM a citation graph" is the null model. We predict that what makes a plan resemble an expert's is the structure that is surprising given hub papers and field size, not the structure that is merely frequent.

Core Claim and Relation to Prior Work

Two systems define the current state of survey generation, and both motivate this work by leaving structure on the table. SurveyMaster (Automating Cognitive Distillation for Expert-Level Scientific Literature Synthesis) expands seed surveys into a topic tree. It then picks fewer than 5% of papers as "core" anchors using citation count and venue prestige. AutoSurvey2 (AutoSurvey2: Empowering Researchers with Next Level Automated Literature Surveys) retrieves about 1,500 candidates and asks an LLM "architect" to write an 8-section outline. It fills each section by dense-vector retrieval. Its graph is a software workflow, not a model of the field. Both systems treat the literature as a ranked or clustered list. We keep their retrieval and writing stages and replace only the planner.

Two further lines of work supply parts of the method. General planners such as STORM (Assisting in Writing Wikipedia-like Articles From Scratch with Large Language Models) build outlines from simulated question-asking before writing. Re3 keeps a global outline as a persistent anchor during long generation. Both support the premise that separating "what to say" from "how to say it" helps long documents. Neither grounds the plan in relations between sources. The neuro-symbolic study (Scaling Scientific Knowledge Discovery with Neuro-Symbolic AI and Large Language Models) shows that conditioning GPT-4 on knowledge-graph triples, split into sub-graphs, gave verifiable citations. Plain document retrieval sometimes did worse than no retrieval. That result constrains our design: structured input helps. But it also raises the strongest rival to our idea, which is to hand the LLM the typed graph with no motif statistics. We test that rival directly.

From network science we reuse the statistical logic. Studies of metabolic networks (Exploring local structural organization of metabolic networks using subgraph patterns) show that raw subgraph counts mislead because hubs generate dense local structure by themselves. They measure significance with Z-scores against degree-preserving randomizations. Subgraph ratio profiles, normalized vectors of over- and under-representation used as fingerprints in Evidence to support common application switching behaviour on smartphones, let us compare the structure of different fields. Work on higher-order ranking (Ranking Users in Social Networks With Higher-Order Structures) motivates scoring papers by the patterns they take part in rather than by first-order links. Regulatory-network work (Architecture of the human regulatory network derived from ENCODE data) supplies the feed-forward loop, the lineage-shaped three-node pattern we reinterpret for scientific ideas.

We make no claim that motifs in citation graphs "compute" anything, as the biological literature argues for gene networks. Our claim is narrower and testable. The enriched typed patterns in a topic's citation graph should predict how human experts group references in surveys. They should predict this better than citation counts, embedding clusters, or raw pattern frequency, and plans built from them should yield more analytical surveys.

Research Program

The pipeline has three stages.

Typed citation graph. For a survey topic, we take the roughly 1,500-paper candidate pool that AutoSurvey2-style retrieval produces. We extract the citation links among these papers and the sentence around each citation. An LLM labels each link with one of a small set of functions: extends, uses-as-tool, compares-against, contradicts, or background. We audit a sample of labels by hand, because labeling noise is the most likely point of failure.

Time-respecting null model. Standard degree-preserving edge swaps are wrong for citation graphs. They create citations to papers published later, and those make certain patterns look "enriched" simply because the randomized copies are impossible. The null model therefore swaps only edges whose endpoints preserve publication order. It also preserves each paper's in- and out-degree and the proportion of each citation type. For each typed three- and four-node pattern class $m$, we compute

$$Z_m = \frac{N_m - \mu_m}{\sigma_m},$$

where $N_m$ is its count in the real graph and $\mu_m, \sigma_m$ are the mean and standard deviation over the randomized copies. The result is the topic's ratio profile. We use these enriched classes to choose narrative templates. A typed feed-forward loop (A extends into B; A and B both used by C) becomes a lineage paragraph. A fork (B and C both compare against A and contradict each other) becomes a "competing approaches" paragraph. A bridge becomes a synthesis paragraph. Class-level enrichment says which kinds of story this field has.

Instance selection and plan assembly. Class-level enrichment does not say which specific instances to write about. Each instance is scored by how much more often its papers co-occur in enriched patterns than their degrees predict under the null. This is a degree-corrected participation score that avoids simply re-selecting hubs. Overlapping high-scoring instances are merged into paragraph units. Units are grouped into sections by the connected pattern structure among them. Each unit becomes a plan entry naming its papers, its relation type, and the citation sentences that justify the relation. An unchanged writer model expands each entry. Every relational claim the writer makes can therefore be checked against a labeled citation sentence.

Scale is modest. A 1,500-node graph with a few thousand internal edges and 1,000 randomized copies for three- and four-node patterns is routine. Efficient canonization methods such as the resolving-set approach in Detection of Network Motif Based on a Novel Graph Canonization Algorithm from Transcriptional Regulation Networks are available if we extend to larger patterns.

Baselines

All generation baselines share one retrieval pool, one writer model, one decoding budget, and one tuning budget of roughly equal validation topics per planner. Only the planner differs.

Naive reference. A long-context model receives all candidate abstracts in a single prompt and writes the survey with no explicit plan. This is the reference point any structured system must clear. SurveyMaster's paper reports that the strongest baseline it compared against reached 3.63 Fact Correctness and 3.62 Citation Precision on SciSurveyBench.

Metadata backbone. This is our re-implementation of SurveyMaster's core-paper selection: the top 5% by citation count and venue, with a topic tree from seed surveys, feeding the shared writer. SurveyMaster itself reports 4.84 Fact Correctness, 4.49 Citation Precision, and 4.06/5 Critical Analysis on SciSurveyBench (Automating Cognitive Distillation for Expert-Level Scientific Literature Synthesis). It is the strongest existing method. We run the released system if available and also our planner-swapped version, so that we can separate planner effects from writer effects. Beating it means a statistically reliable gain on Critical Analysis with Fact Correctness and Citation Precision no lower. Near-ceiling fact scores leave little room for improvement there.

Embedding-cluster planner. This is AutoSurvey2's approach: dense-vector clustering plus an LLM architect.

PageRank planner. Sections are built around top-PageRank papers in the same citation graph.

Raw typed graph. The LLM planner receives the full typed citation graph as triples and plans freely. This is the neuro-symbolic setting extended to planning.

Key ablations. The first ablation is identical to our planner but ranks patterns by raw count instead of Z-score. The second uses a standard, non-time-respecting null model. The third uses an untyped graph. Together they isolate whether the null model, time order, and citation typing each matter.

Why not just…?

Why not just rank by citation count or PageRank? These methods find influential papers but say nothing about how papers relate. A survey section is a relation among papers, not a list of them. Higher-order ranking work (Ranking Users in Social Networks With Higher-Order Structures) shows that first-order walks miss modular roles. The reference-grouping test below measures directly whether hub-centered grouping recovers expert sections worse than motif grouping.

Why not just cluster embeddings? Cosine similarity is symmetric and blind to direction. A refutation and an extension of the same method sit close together in embedding space, yet an expert writes them into different narratives. This is AutoSurvey2's planner and is run as a baseline. If it matches us on grouping agreement but loses on Critical Analysis, the value of typed relations is confirmed.

Why not just give the LLM the typed citation graph? This is the real threat. The neuro-symbolic results (Scaling Scientific Knowledge Discovery with Neuro-Symbolic AI and Large Language Models) show that structured triples already help. A strong LLM might find lineages and forks unaided. Our bet is that, without a null model, it will over-weight hub-dense regions, as raw counts do in Exploring local structural organization of metabolic networks using subgraph patterns. The raw-typed-graph baseline and the raw-count ablation settle this. If either matches the Z-score planner, the statistical machinery is unnecessary, and we would report that the contribution is citation typing alone.

Why not just use a long-context model? Unplanned long generation drifts, and SurveyMaster's gains came from structure rather than prompting. The naive reference measures this on our data rather than assuming it.

Proposed Experiments

Diagnostic 1: reference grouping in human surveys. This test comes first, is cheap, and decides whether to proceed. SciSurveyBench provides 100 peer-reviewed gold surveys across 10 disciplines. For each survey, we rebuild the typed citation graph over its reference list and ask each planner to partition the references into sections and paragraphs. We score agreement with the human partition using adjusted Rand index. If the Z-score planner does not beat the PageRank, embedding, and raw-count partitions here, the generation study is not worth running.

Diagnostic 2: related-work paragraphs. Multi-XScience (Multi-XScience: A Large-scale Dataset for Extreme Multi-document Summarization of Scientific Articles, code) pairs about 4.4 cited references with a human related-work paragraph. Its authors name missing citation-graph structure as a limitation. We ask whether the reference sets that humans place in one paragraph score higher on motif participation than equal-size random subsets of the same citing paper's references. This requires reconstructing full reference lists from the arXiv sources. For generation on this benchmark, the reported Pointer-Generator ROUGE-L of 30.63 is the published reference point. No reported result was found on Lacuna for current instruction-tuned LLMs on this benchmark.

Main study: survey generation. We generate surveys on SciSurveyBench's 100 tasks with every planner. We score them with its 8-criterion LLM-as-judge protocol, using two judge models to check for judge bias. We add two structural measures. The first is the fraction of sentences that assert a relation between two or more cited works and are supported by a labeled citation sentence. The second is paragraph-level reference-grouping agreement with the gold survey. A blinded expert rating of Critical Analysis on a subset of 20 topics guards against judge artifacts.

Decisive checks. Three results would support the claim. The Z-score planner must beat the raw-typed-graph and raw-count variants on grouping agreement and Critical Analysis. The time-respecting null must beat the standard null. Human raters must prefer our surveys' comparative sections. The claim is refuted if the raw typed graph given to an LLM matches us, which would mean typing, not surprise, carries the gain. It is also refuted if enrichment profiles look alike across all 10 disciplines, which would mean the motifs reflect citation habits rather than field-specific intellectual structure. We report the ratio profiles per discipline so the second possibility is visible.