Most work on influence in social networks asks a yes-or-no question: which $k$ people should we "seed" so that a message spreads as far as possible? Real campaigns, and real defenders against misinformation, spend money differently. They split a budget into continuous amounts across many people and many rounds, and they do it while an opponent is doing the same thing. A small theory paper has shown what good continuous allocations look like against a fixed opponent on idealized graphs. Against a fixed opponent on regular graphs, the best defense is to fund a protective ring around the people the opponent targets ("shielding"). The best attack depends on the budget: outspend the opponent on its own targets when richer ("shadowing"), and retreat to uncontested ground when poorer. Nobody knows whether these postures survive on real, uneven networks against an opponent that learns.
This note proposes to find out, and to settle when a learned continuous policy is worth having at all. A policy here is a network that maps the current pattern of opinions on the graph to a budget vector over nodes. The setting is a multi-round voter model, where each person repeatedly copies the opinion of a neighbor or of an outside campaigner, on real social graphs. The key design choice is to avoid the usual approach of continuous-control reinforcement learning, which learns a "critic" (a network that scores how good an action was) from scratch. In this model the expected opinion dynamics can be computed exactly and differentiated. So the actor is trained by backpropagating through those exact dynamics, and the only learned component is a value estimate for what happens beyond the planning window. The central empirical question is sharp: does this learned policy beat simply re-solving the allocation by gradient ascent every round? The answer is tested both against a fixed opponent and against one that adapts.
Core Claim
Two claims, one methodological and one scientific. The methodological claim is that in competitive continuous influence problems, a hybrid actor-critic is more sample-efficient and more stable than a DDPG-style actor-critic whose critic is learned from simulated rollouts. In the hybrid, the short-horizon part of the critic is the exact differentiable expected-opinion model and only a terminal value is learned. If the hybrid also outperforms per-round re-optimization, the advantage should appear specifically against adaptive opponents and when one policy must be deployed on unseen graphs. Against a fixed opponent, we expect per-round re-optimization to be nearly as good, and we will say so if it is. The scientific claim is that on heavy-tailed graphs the shielding posture fragments. Budget should flow to the specific boundary nodes that carry many edges into the opponent's region rather than to the whole frontline. The budget-dependent switch between shadowing and avoidance should then shift in a way the regular-graph mean-field theory does not predict.
Research Program
The environment extends the model of Shielding and Shadowing. That paper places binary-opinion agents on a weighted graph and has two outside controllers push on each node $i$ through continuous coupling strengths $a_i, b_i \ge 0$ with total budgets. It finds Controller A's best response by offline gradient ascent and explains it with a three-group mean-field approximation on $K$-regular graphs. We keep the continuous couplings and make the game sequential. Over $H$ rounds, each controller chooses a per-round allocation from a total budget it can spend across time. In each round, every node copies a neighbor with probability proportional to edge weight, or copies a controller with probability proportional to that controller's coupling. Writing $x_i^t$ for the probability that node $i$ holds opinion A after round $t$, the copy rule gives an expected update that is linear in the opinions once allocations are fixed:
$$x_i^{t+1} = \frac{\sum_j w_{ij} x_j^t + a_i^t}{d_i + a_i^t + b_i^t},$$
where $w_{ij}$ is the edge weight, $d_i = \sum_j w_{ij}$ is node $i$'s weighted degree, and $a_i^t, b_i^t$ are the two controllers' couplings in round $t$. This recursion is exact for the expectation and differentiable in the allocations. It plays the role that exact gradients from reverse influence sampling play in Gradient Method for Continuous Influence Maximization with Budget-Saving Considerations. That paper uses such gradients for a single static proximal-gradient solve; we reuse them to train a policy. Reverse influence sampling estimates spread by sampling, for random target nodes, the set of nodes that could have reached them. Realized opinions on finite graphs are stochastic, so the agent observes sampled states rather than expectations.
The actor is a graph neural network over node features: current opinion, degree, and the opponent's last allocation. It outputs a softmax over nodes, scaled by a learned per-round spending fraction. This satisfies the budget constraint without a projection layer that would bend gradients. It is trained by differentiating the expected vote share through $L$ unrolled rounds of the recursion, plus a learned terminal value $V_\phi$ for the remaining rounds. $V_\phi$ is fit on sampled trajectories. The opponent is trained the same way. Training alternates in a double-oracle loop borrowed from Finding Needles in a Moving Haystack. There, DDPG-MIX represents a defender's choice as a continuous distribution over alert types and computes an equilibrium against an adaptive attacker, with no network diffusion. We take its equilibrium machinery and replace its learned critic with a model-based one.
To address the "paradox of sensitivity" from Adaptive Budget Allocation for Maximizing Influence of Advertisements, we add an explicit commitment interval. That paper shows greedy policies that react to every noisy feedback signal underperform policies that commit budget in chunks. Here the policy may only change its allocation every $T$ rounds, and $T$ is swept. A second environment checks that the method is not tied to the voter model. It uses Independent Cascade spread, in which each newly activated node gets one chance to activate each neighbor, with continuous investment levels as in Continuous Influence Maximization. The differentiable model there is the reverse-influence-sampling gradient estimator.
Analysis has two parts. First, we fit allocations against graph structure, regressing node budget on distance to opponent targets, number of edges into the opponent region, and degree. Second, we compare the learned shielding and shadowing boundaries with the mean-field predictions on $K$-regular graphs. On those graphs the method should reproduce the published postures; that serves as a correctness check before moving to heterogeneous graphs.
Baselines
The naive reference is uniform allocation across nodes and rounds, along with degree-proportional allocation. Fractional Budget Allocation for Influence Maximization reports that such centrality heuristics are often suboptimal because they ignore network dynamics. DDPG-MIX's reported 50% reduction in defender loss relative to uniform random allocation on CICIDS2017, from Finding Needles in a Moving Haystack, is the only figure in the file for a continuous actor-critic beating uniform allocation. It comes from a different domain, so it indicates the scale of gain to expect, not a target.
The strongest competitor is receding-horizon re-optimization. At every round, solve for the allocation by gradient ascent through the same exact recursion over the same $L$-round window, taking the opponent's current allocation as fixed. This is the method of Shielding and Shadowing applied online. For the Independent Cascade environment, the corresponding baseline is ProxGrad-RIS re-run each round, from Gradient Method for Continuous Influence Maximization with Budget-Saving Considerations. It uses the same model and the same gradients and differs only in having no learned policy and no learned future value. For fairness it gets the same unroll length, the same total gradient steps per decision as the actor's training budget divided across evaluation episodes, and its own tuned learning rate. The only reported number for this family in the file is ProxGrad-RIS's 30x speedup over stochastic-gradient ProxGrad on the 679-node DM network. No reported result was found on Lacuna for vote share or spread from per-round re-optimization in a competitive multi-round setting. The proposal beats it if it achieves higher equilibrium vote share against the adaptive opponent, with non-overlapping confidence intervals across seeds. Matching it at a fraction of the per-decision compute on unseen graphs would be a weaker but still useful win.
The discrete-action competitor is DRIM from Beyond Binary Opinions. It uses PPO to pick discrete seed nodes against a learned competitor and reports 57% to 88% higher influence than STORM and C-STORM on FBN, FBPN and URV. It will be run with its budget converted to an equal number of unit couplings per round and trained for the same number of environment steps.
Two ablations isolate the key ingredient. The first is a DDPG agent with a fully learned critic and the same actor architecture and data budget, which tests whether the exact model matters. The second is the hybrid without $V_\phi$ (pure $L$-step unroll), which tests whether the learned future value matters.
Why not just…?
Why not just re-solve the allocation by gradient ascent every round? This is the real threat. Against a fixed opponent we expect it to come close to the learned policy, because the recursion is exact and the problem per round is smooth. The proposal stands or falls on two experiments. The first is the adaptive-opponent game, where a myopic re-solver treats the opponent as fixed and should be exploitable once the opponent learns to anticipate it. We will measure this exploitability directly by training a best-response opponent against each frozen defender. The second is deployment on held-out graphs, where the learned actor needs one forward pass. If re-solving matches the hybrid on exploitability, we will report that learned policies are unnecessary for this problem class. That is a useful negative result.
Why not just discretize the budget and use DRIM-style discrete reinforcement learning or a greedy seed method like IMM? Discrete seed choice cannot express shielding as fractional pressure spread over a frontline. Fine discretization blows up the action space, as argued in Energy Management in a Cooperative Energy Harvesting Wireless Sensor Network. Discrete reinforcement learning is still a credible empirical competitor: DREIM in Finding Influencers in Complex Networks activated about 26,700 nodes against IMM's 20,180 on Facebook at $k=100$. The head-to-head comparison against DRIM under matched total budget settles this.
Why not just partition into communities and split the budget greedily? Community-IM in A Community-Aware Framework for Social Influence Maximization is fast, at 3,782 s versus 17,359 s for CELF++ on Facebook. But it ignores edges between communities, and those are exactly where the shielding results say budget should go. We include it as a baseline in the single-party Independent Cascade environment. We also directly measure the fraction of learned budget placed on cut edges between Louvain communities; a large fraction would explain any gap.
Proposed Experiments
Graphs come from the file: random $K$-regular ($K=3,4$), Watts-Strogatz and Barabási-Albert graphs at $N=1{,}000$ as in Shielding and Shadowing, and the real networks URV email (1,133 nodes), Facebook FBN (4,039), FBPN (22,470), the Twitter follower graph (23,370), and Epinions. Actors are trained on synthetic graphs and the smaller real ones and evaluated on the rest. Opponent budgets range from $b_{\max}/a_{\max}=0.5$ to $2$ with opponent target fraction $0.1$. The metrics are final expected and realized vote share; exploitability, meaning the loss in vote share when a freshly trained opponent best-responds to the frozen policy; wall-clock time per decision; and training samples to reach 95% of final performance.
The first decisive check is a sanity check: on $K$-regular graphs against a passive opponent, all gradient methods should recover shielding and the shadowing/avoidance switch at $a_{\max}=b_{\max}$. Failure here would invalidate the environment. The second check is the exploitability comparison of hybrid versus receding-horizon re-solving against adaptive opponents on FBN and Twitter; this is the result that justifies or kills the learned policy. The third check is the learned-critic DDPG ablation: we expect the exact-model critic to train in far fewer samples and avoid the critic overestimation problems noted for high-dimensional continuous control. The fourth check is the commitment-interval sweep, which tells us whether the paradox of sensitivity reappears in continuous competitive control. Finally, the structural regression on Barabási-Albert and real graphs tests the scientific claim. Shielding is predicted to concentrate on high cut-degree boundary nodes, and the switch point between shadowing and avoidance is predicted to move away from equal budgets as degree heterogeneity grows.
References
- Shielding and Shadowing: A Tale of Two Strategies for Opinion Control in the Voting Dynamics
- Gradient Method for Continuous Influence Maximization with Budget-Saving Considerations
- Finding Needles in a Moving Haystack: Prioritizing Alerts with Adversarial Reinforcement Learning
- Adaptive Budget Allocation for Maximizing Influence of Advertisements
- Continuous Influence Maximization
- Fractional Budget Allocation for Influence Maximization
- Beyond Binary Opinions: A Deep Reinforcement Learning-Based Approach to Uncertainty-Aware Competitive Influence Maximization
- Energy Management in a Cooperative Energy Harvesting Wireless Sensor Network
- Finding Influencers in Complex Networks: An Effective Deep Reinforcement Learning Approach
- A Community-Aware Framework for Social Influence Maximization