Research groups that pool data for a shared model face two demands that pull against each other. They want their individual update to stay hidden, because raw model updates can leak the underlying patient records or unpublished data. They also want credit that reflects what their data actually did for the model. Secure aggregation, a cryptographic trick that lets a server learn only the sum of everyone's updates, delivers the first and blocks the second. Existing ways to score contributions under secure aggregation buy back credit by evaluating the model "with and without" each participant. Each such evaluation adds another linear equation about the hidden updates, and a server can eventually solve those equations.
We propose a different way to assign credit. Each participant commits cryptographically to its update before training. After the commitments are fixed, a neutral validation holder publishes a single "useful direction": the gradient of validation loss at the released global model. Each participant then reports one number, how far its accumulated update moved the model along that direction, and proves in zero knowledge that the number matches its committed update. The server learns a handful of scalar projections per participant per training run instead of whole vectors or counterfactual models. Participants cannot inflate the number they report. A participant contributing nothing scores exactly zero, and one contributing pure noise scores zero in expectation. Credit is released once, at the end of training, rather than round by round. That timing removes the early-access channel behind researchers' fear of being scooped.
Why this gap is real
The incentive literature explains why credit matters. Data reuse is extremely concentrated: only 3.5% of 414,022 datasets in an OpenAlex snapshot were ever cited, and the top 10% take nearly 75% of citations (Krause et al.). Mandates have not fixed this. "Available on request" statements went unanswered or were declined 93% of the time (Watson et al.), and journal policies in specialist fields mostly encourage sharing rather than require it (Neal et al.). Economic models show that sharing is rational only when a contributor's marginal value is recognized and competitive exposure is limited (Tsoy and Konstantinov). These papers motivate the proposal. They establish that individual, manipulation-resistant credit is the missing piece. They do not tell us how to compute it under encryption.
The secure-aggregation literature constrains the design. So et al. show that when the set of participants summed in each round varies, the server collects linear equations and recovers individual updates within $O(N)$ rounds. Their fix is to keep clients in fixed batches of size $T$ that always participate together. Contribution evaluation, however, wants exactly the opposite: comparisons with and without a participant.
Three bridging papers sit in this tension. Jaramillo Velez et al. propose two scores computable under secure aggregation. Fair-Private relies on clients evaluating models with and without their own update. Everybody-Else has clients evaluate the others' impact; it resists self-inflation but gives up the null-player property, so a do-nothing participant can earn a nonzero score. Ma et al. compute Shapley values over groups and split rewards evenly within groups, which lets free-riders hide inside good groups. Zhang et al. (VeCoSA-FI) use Pedersen commitments and Sigma protocols to make aggregation verifiable, but they score credibility from latency and attendance rather than from data value. Non-private methods show what fine-grained credit can look like. Tastan et al. compare per-client classifier-head updates to the global update by cosine similarity, and Qiao et al. build reward schemes with fidelity-graded outputs. Both, however, require the server to see individual parameters.
Our proposal reuses VeCoSA-FI's cryptographic toolkit for a different job: proving data value rather than attendance. It keeps So et al.'s batch partitioning intact rather than violating it.
Core Claim
A first-order contribution score can be computed and verified under fixed-batch secure aggregation, with leakage that can be counted. Write $x_i$ for participant $i$'s update summed over a training phase, and $g$ for the validation-loss gradient at the released model, published only after all commitments are fixed. The score is
$$s_i = -\langle g, x_i \rangle,$$
the first-order estimate of how much participant $i$'s update lowered validation loss.
Because $s_i$ is linear in $x_i$, a participant holding a Pedersen vector commitment to $x_i$ can prove with a standard Sigma protocol that its reported $s_i$ is correct, without revealing $x_i$. The commitments are homomorphic, meaning they can be combined in the same way the hidden vectors are added. The server can therefore check that the product of the commitments matches the masked sum it aggregated, which ties the scored update to the update actually used in training.
Over $K$ scoring phases, the server observes the batch-level sums (still protected by So et al.'s $T$-client guarantee) plus $K$ scalar projections per client. With $K$ in the tens and model dimension $d$ in the millions, the observations remain far from determining any individual update. Calibrated noise added to each $s_i$ gives a formal differential-privacy bound on top.
The claim is that this score ranks contributors about as well as Fair-Private and Everybody-Else. It should also beat both on manipulation resistance and null-player behavior, and it should leak strictly less across rounds.
Research Program
The work has three parts.
Leakage accountant. The first part extends So et al.'s rank argument to cover evaluation queries as well as training sums. Given any schedule of aggregates, counterfactual evaluations, and scalar disclosures, the tool computes the smallest number of clients mixed in any linear combination the server can isolate. It also reports how much of each client's update is determined, measured as the projected subspace dimension. We will run existing schemes through it, Fair-Private, Everybody-Else, and GroupSV under realistic dropout, to measure whether and when their queries break multi-round privacy. This analysis is a contribution on its own: the bridging papers evaluate leakage within a single round.
Protocol. The second part builds the protocol itself. It combines pairwise additive masking with batch partitioning for training, Pedersen vector commitments to quantized updates, commit-then-challenge publication of $g$ by a validation holder, and Sigma-protocol proofs of the reported projections. Scores are released only after training, optionally converted into a contribution certificate that a consortium could attach to authorship or dataset citations. The certificate links to the citation problem Krause et al. document, but evaluating it is out of scope here.
Commitment cost is the main engineering risk. Zhang et al. report about 74 seconds of server time for 1,000 clients and $10^5$ parameters, so we target cross-silo settings of at most 50 institutions. We will reduce cost by committing to random low-dimensional sketches of each update, whose validity is itself verified, and measure how much this sketching loses in score quality.
Gaming analysis. The third part studies how participants can game the score. A first-order score rewards alignment with $g$. A strategic participant who can predict $g$ could therefore try to shape its update toward it without contributing useful data. Because $g$ is computed on held-out data after commitment, such shaping should work only to the degree that it also improves validation loss, which is the behavior we want to reward. We will test this directly with adaptive attackers. These attackers estimate $g$ from public data and craft updates to maximize $s_i$, and we measure their score gain against their actual effect on held-out loss.
Baselines
Naive reference. The do-nothing reference splits credit equally, or in proportion to self-reported dataset size. It requires no extra disclosure and sets the floor for both ranking quality and manipulability: anyone can inflate a reported size.
Strongest competitor. The strongest competitor under the same privacy model is the pair of Fair-Private and Everybody-Else scores from Jaramillo Velez et al.. They are the only published scores designed to run under secure aggregation in linear time. They have been validated on the same medical benchmarks we use: ISIC2019, PatchChameleon, Brain-MRI, and Dirichlet-split CIFAR-10. That paper reports "high" Spearman, Kendall and Pearson correlations with retraining-based Shapley values. There is no reported numeric result for these correlations on Lacuna, so we will reproduce both scores from the paper's description. They will run on identical client partitions, the same model, the same number of rounds, and the same tuning budget for any hyperparameters. Retraining Shapley will be computed exactly for six to ten clients as ground truth.
We beat this baseline if three things hold. Our rank correlation must be within noise of theirs, or better. Our score inflation under manipulation must be clearly lower than Fair-Private's. And null and noise participants must score near zero, where Everybody-Else does not. All of this must hold at strictly lower leakage as measured by the accountant.
Group-level and non-private references. Ma et al.'s GroupSV, run with the same batches as groups, represents the alternative of giving up individual credit. ShapFed's class-specific Shapley scores from Tastan et al. serve as a non-private upper reference. That paper reports a fairness correlation of 0.84 for ShapFed versus 0.63 for FedAvg on Fed-ISIC2019. We do not need to beat 0.84; approaching it without server access to individual updates would be the result.
Ablations. Four ablations isolate the key ingredients. Removing the commitments and proofs, so projections are self-reported, isolates verifiability. Publishing $g$ before commitment isolates commit-then-challenge. Disclosing projections every round instead of once per phase isolates the leakage budget. Dropping batch partitioning isolates its interaction with scoring.
Why not just…?
Why not just use Fair-Private or Everybody-Else as they are? Both are serious options, and on ranking quality alone they may match us. Their weak points lie elsewhere. Fair-Private depends on self-evaluation, which Jaramillo Velez et al. themselves identify as an attack surface. Everybody-Else drops the null-player property by design. Neither has been analyzed for leakage across rounds, which So et al. show is where secure aggregation actually fails. Our leakage accountant and manipulation experiments test exactly these points. If they show Fair-Private's and Everybody-Else's queries stay underdetermined under batch partitioning, and their manipulation gain is small, the case for our protocol narrows to null-player behavior and verifiability. We would report that honestly.
Why not just pay groups, as GroupSV does, and keep individuals fully hidden? Group payment is the most private option and a real threat to the proposal if free-riding turns out to be rare. The evidence against it is structural: equal splits within a group pay a noise-injecting member the same as a strong one. We will test this by planting free-riders, label-noise clients, and zero-update clients inside high-value batches. Detection is measured as the area under the ROC curve (AUC) for separating planted clients from honest ones. If GroupSV's even split costs honest contributors little in realistic consortia, individual scoring is not worth its overhead.
Why not just require deposit in a trusted repository or clean room and skip cryptography? That route already exists and has not worked. Mandated sharing produced a 93% failure rate for on-request data and a 3% results-reporting rate in a COVID-19 trial cohort (Watson et al.). Removing identifiers also fails for network-structured data (Neal et al.). The competition models of Tsoy and Konstantinov explain why a dominant data holder would not hand over raw assets. This proposal cannot test that behavioral claim. Its narrower aim is to make the cryptographic option offer what repositories cannot: protection and credit at once.
Why not just add differential-privacy noise to leave-one-out scores? Noise addresses leakage but not dishonesty: a self-reported leave-one-out loss is still unverifiable. The "no proofs" ablation is exactly this alternative, and we expect it to show large score inflation under strategic reporting.
Proposed Experiments
The experiments use Fed-ISIC2019 with its six natural centers, and ISIC2019, Brain-MRI, PatchChameleon and CIFAR-10 under Dirichlet splits, matching Jaramillo Velez et al. CIFAR-100 with batch sizes $T=3$ and $4$ allows direct comparison with So et al.'s setting.
Score quality. We measure Spearman and Kendall correlation with exact retraining Shapley. This test has a clear failure condition. If our first-order score correlates clearly worse than Fair-Private on the extreme non-IID Fed-ISIC2019 split, where updates can be large and the linear approximation may break, the core claim fails. In that case we would test scoring at more frequent phases before giving up.
Manipulation. We measure the score gain of adaptive attackers who fabricate reports, which proofs should rule out, or craft updates aligned with an estimated $g$. Gain is compared to their true effect on held-out loss.
Privacy. We combine the accountant's isolation results for every scheme with an empirical inversion attack that reconstructs individual inputs from everything the server sees. Reconstruction quality is reported as MSE and cosine similarity. As a reference point, VeCoSA-FI reports MSE above 1.9 and cosine below 0.1 against its adversary with 40% malicious nodes.
Cost and utility. We report final model accuracy against unmasked FedAvg and per-client and per-server time for commitments and proofs, with and without sketching.
The decisive check is a single comparison on Fed-ISIC2019 with planted free-riders and noise contributors. The proposal succeeds if it matches Fair-Private and Everybody-Else on Shapley rank correlation and catches free-riders that GroupSV pays. It must also give null participants near-zero credit and resist fabricated reports, while the accountant shows no individual update becomes recoverable over the full training run.
References
- Measuring Data Re-Use In OpenAlex by Researchers, Institutions, and Countries
- Delivering on NIH data sharing requirements: avoiding Open Data in Appearance Only
- Recommendations for sharing network data and materials
- Strategic Data Sharing between Competitors
- Securing Secure Aggregation: Mitigating Multi-Round Privacy Leakage in Federated Learning
- Private and Robust Contribution Evaluation in Federated Learning
- Transparent Contribution Evaluation for Secure Federated Learning on Blockchain
- VeCoSA-FI: A verifiable and collusion-resistant secure aggregation protocol in federated learning with fair incentive mechanism
- Redefining Contributions: Shapley-Driven Federated Learning
- Collaborative Causal Inference with Fair Incentives