When an experiment records only a few dozen or a few hundred events, the summary numbers physicists extract (means, variances, angular moments of a distribution after correcting for detector blurring) fluctuate so much that they can stop describing any probability distribution at all. A variance comes out negative, a fourth moment falls below the square of the second, a reconstructed density goes negative somewhere. We propose to estimate these moments with a convex program that only searches over moment vectors that some real distribution could have produced. Group symmetries of the detector or of the event are used for two purposes. They shrink that program to a size that can be solved, and they cut the number of free quantities the scarce data must pin down.
The opening matters because current unbinned "unfolding" methods are fast and flexible but give no such guarantee. Unfolding here means correcting measured distributions for detector distortion without first sorting events into histogram bins. These methods are also least reliable exactly where statistics are thinnest: high-momentum tails, rare channels, and early-run data. A method that is convex, returns a certificate, and is tied to the symmetry would give physicists moment estimates and intervals that are physical by construction. It would also test a real question: whether symmetry reduction, so far used as a pure computational trick, also buys statistical power.
Core Claim
Unbinned moment unfolding can be posed as a semidefinite program, an optimization over matrices constrained to have no negative eigenvalues. Two ingredients are combined. The first is Lasserre's characterization (Borel measures with a density on a compact semi-algebraic set): a truncated moment vector $y$ comes from a measure on a set $K=\{z: g_j(z)\ge 0\}$ only if its moment matrix $M_d(y)$ and localizing matrices $M_{d-d_j}(g_j y)$ are positive semidefinite. The second is Gatermann's block-diagonalization (Symmetry groups, semidefinite programs, and sums of squares). When the truth-level distribution is invariant under a group $G$, both $M_d(y)$ and the estimation problem can be restricted to the $G$-invariant subspace. In that subspace the matrices split into small independent blocks.
Our claim has three parts. The first is computational: the reduction makes multivariate moment unfolding tractable at degrees where the unreduced moment matrix is out of reach. The second is statistical and is the part that is new. Under scarcity, the reduced and constrained estimator has lower error and better interval coverage than unconstrained or post-hoc corrected estimators. The gain is largest at small event counts $N$, where the positivity constraints are actually active.
The intuition for the statistical claim is simple. With $p$ free moments estimated from $N$ events, the unconstrained estimation error grows roughly as $\sqrt{p/N}$, the usual parameter-counting rate for least squares. Restricting to the invariant subspace replaces $p$ by $p_G$, which for a finite group is roughly $p/|G|$. The cone constraint then removes a further, data-dependent part of the error when the noisy estimate would otherwise fall outside it.
The third part is the step beyond point estimation. The same program returns sharp upper and lower bounds on any linear quantity of interest, such as the fraction of events in a tail region, over all valid distributions consistent with a Poisson confidence region for the measured moments. These are intervals that cannot contain unphysical distributions.
One point separates this from the symmetry-in-optimal-transport line. SG-ROT (Optimal Transport with Symmetry Groups) discovers finite-group orbits from distance patterns and states that continuous groups remain open. Our setting avoids that obstacle. A continuous group such as azimuthal rotation $SO(2)$ acts on polynomials of bounded degree through a finite-dimensional representation, so the reduction is exact Fourier-mode bookkeeping and needs no orbit discovery.
Research Program
The first component is the forward model. Unfolding requires a map from truth-level moments $y$ to expected detector-level moments. For additive smearing independent of the truth value this map is exactly linear: $\mathbb{E}[z'^a]$ is a fixed combination of truth moments $y_b$, with $b \le a$. For realistic detector simulation the map is not exactly linear. We will therefore estimate a linear operator $A$ from paired simulated events by regressing detector-level monomials on truth-level polynomials up to degree $2d$. We report the regression residual as an explicit, measured model error and do not hide it.
The estimator solves
$$\hat y=\arg\min_{y\in V^G}\ (A y-\hat m)^\top \Sigma^{-1}(A y-\hat m)\quad \text{s.t.}\quad M_d(y)\succeq 0,\ M_{d-d_j}(g_j y)\succeq 0,$$
where $\hat m$ is the vector of empirical detector-level moments, $\Sigma$ is its estimated covariance (built from the higher empirical moments, which carry the Poisson counting noise), and $V^G$ is the $G$-invariant moment subspace. The quadratic objective is convex, so this is a semidefinite program with a certificate of optimality. Interval bounds on a target functional $\langle c, y\rangle$ come from minimizing and maximizing it subject to the same constraints plus the confidence ellipsoid $(Ay-\hat m)^\top\Sigma^{-1}(Ay-\hat m)\le \chi^2_{\alpha}$.
The second component is the symmetry machinery. We will build the symmetry-adapted bases for three groups that arise physically. The permutation group $S_m$ acts on $m$ identical particles described by $k$ features each, so the moment matrix lives in $mk$ variables and grows as $\binom{mk+d}{d}$. The groups $SO(2)$ and $O(2)$ act on azimuthal coordinates. A dihedral group $D_k$ describes a segmented cylindrical detector. Gatermann's results (Symmetry groups, semidefinite programs, and sums of squares) show the scale of reduction available. A $715\times715$ Gram matrix for a degree-8, 10-variable invariant polynomial collapsed into one $55\times55$ block, forty-five $10\times10$ blocks, and 210 scalar constraints. Blocks of that size are solvable in seconds with standard interior-point solvers. For larger instances we will borrow the low-rank, matrix-free approach and the Lanczos-based eigenvalue certification of Matrix-Free GPU Semidefinite Programming for Quantum Ordered Search at the k=6 Frontier. Unlike that work's convolution kernels, which are tied to near-circulant structure, our kernels operate block by block in the symmetry-adapted basis.
The third component is analysis of where the approach stops being tractable. The matching problem has no small symmetric SDP shows that symmetry cannot rescue every problem. Its lower bound concerns approximating a combinatorial polytope, while our relaxations have fixed degree over continuous variables. We will nonetheless measure block sizes as $m$ and $d$ grow and report where the largest block exceeds a fixed memory budget. That turns the tension the file identifies into a measured scaling curve rather than an assertion.
The fourth component is detecting when symmetry fails. Real detectors break symmetry. We will estimate the non-invariant part of the detector-level moments, using the Reynolds residual $\hat m - \mathcal{R}_G(\hat m)$, and test it against its Poisson noise level before imposing $V^G$. When the test rejects symmetry, the method falls back to the unreduced or partially reduced program. This rule is part of the method, not a hidden choice.
Baselines
The do-nothing reference is the unconstrained linear inversion. We take the empirical detector-level moments, apply the generalized least-squares inverse of the same estimated operator $A$, and use no positivity constraint and no symmetry. It uses identical inputs to our estimator, so any improvement is attributable to the constraints. Beating it means lower truth-moment error at every event count below some threshold. We will also count how often it outputs moment vectors outside the moment cone.
The binned classical reference is Iterative Bayesian Unfolding on histograms of each observable, with moments computed from the unfolded histogram. Its binning and iteration count are tuned on held-out simulation with the same budget we give our degree and regularization choices. OmniFold (OmniFold: A Method to Simultaneously Unfold All Observables) reports matching or improving on this approach for the six jet observables, so we expect it to be weakest, but it is the community default.
The strongest existing method for this specific question is Moment Unfolding (Moment Unfolding). It fits a Boltzmann-weighted reweighting with a discriminator and directly targets moments of jet mass, charge, width, and $z_g$ on the Pythia/Herwig/Delphes benchmark. Its reported advantage is speed: it is about 100 times faster than OmniFold. We found no reported numerical moment error or coverage for it on Lacuna at small event counts, and its guarantees are asymptotic. We will run it with the authors' architecture and a fixed hyperparameter search budget, using the same training simulation and identical subsampled pseudo-data. Because the network is non-convex, it gets ten random seeds per pseudo-dataset. OmniFold followed by computing moments of the reweighted particle-level sample is a second strong baseline. It is run at matched wall-clock budget in the high-statistics regime and at its default configuration in the scarce regime.
Three ablations isolate the key ingredients. The first is our program without symmetry: full moment matrix, positivity only. The second is symmetry without positivity: Reynolds-averaged moments with linear inversion. The third is Moment Unfolding's output projected onto the moment cone after the fact. Beating the third ablation is the result that justifies a joint constrained fit rather than a cleanup step.
Why not just…?
Why not just symmetrize the data and use the existing neural method? Averaging the observed events over the group (the Reynolds operator applied to data) captures the parameter-count reduction for any estimator, including Moment Unfolding. This is a genuine threat to the statistical part of our claim. The symmetry-without-positivity ablation and a symmetrized Moment Unfolding run measure it directly. If symmetrized baselines match our error and coverage, we will conclude that the gain comes from symmetry alone, the semidefinite program contributes only certification, and the claim becomes narrower.
Why not just project a neural estimate onto the moment cone afterward? Projection is a small semidefinite program and repairs feasibility. However, it ignores the measured covariance $\Sigma$, so it moves the estimate in directions the data constrain poorly as freely as in well-constrained ones. It also produces no interval with frequentist meaning. The projected-Moment-Unfolding ablation settles this. If it matches our point-estimate error, we fall back to claiming only calibrated intervals, which projection cannot produce.
Why not just use a general-purpose solver on the full moment matrix, since one-dimensional moment problems are tiny? For a single observable at degree eight, the moment matrix is $5\times5$ and symmetry is irrelevant. We concede that case. The reduction matters for joint moments of several identical particles, or for azimuthally resolved observables. There the unreduced dimension grows combinatorially, as Gatermann's $715\times715$ example shows. The scaling experiment reports the largest block size and solve time with and without reduction, and identifies where the unreduced solver fails our memory budget.
Why not just bootstrap the existing estimator for intervals? Bootstrapping a non-convex estimator at $N\le100$ resamples an already unstable fit, and its intervals can include unphysical moment vectors. We will include bootstrap intervals for Moment Unfolding in the coverage study and let the measured coverage decide.
Proposed Experiments
The first benchmark is the one-dimensional Gaussian smearing toy from Moment Unfolding, where the response is exactly linear in moments. We extend it with a truth distribution placed near the boundary of the moment cone, for example a strongly bimodal mixture. That is where positivity should bind. We sweep $N$ from 30 to $10^5$ events over 1,000 pseudo-experiments per point. We report root-mean-square moment error, the fraction of infeasible outputs, and empirical coverage of nominal 68% and 95% intervals.
The second benchmark is the Z+jet sample used by OmniFold and Moment Unfolding: about 1.6 million events, Pythia 8.243 as simulation, Herwig 7.1.5 as pseudo-data, Delphes 3.4.2, $p_T>200$ GeV, $R=0.4$. We subsample the Herwig pseudo-data to emulate scarcity and also apply a high-$p_T$ selection, which is naturally sparse. We unfold the single-observable moments the prior papers report. We then unfold joint moments of the leading $m$ constituents' momentum fractions and angular distances. These are permutation-invariant under $S_m$, which is where the reduction is exercised. The residual of the linear response fit is reported alongside, because a large residual would mean the forward model, not the estimator, limits accuracy.
The third experiment is a synthetic cylindrical-detector toy with $D_k$ or $O(2)$ symmetry and a small signal on a Poisson background. We compute tail-fraction bounds and compare their width and coverage against an extended unbinned likelihood fit with the correct parametric shapes. The file reports no public unbinned rare-event dataset, so this experiment stays synthetic. EXO-200, BESIII, and similar results serve only as context for the event counts we emulate. A final demonstration on CMS Open Data (Disentangling Quarks and Gluons with CMS Open Data), using its forward and central jet samples, checks that the method runs on real detector data without claiming a new physics result.
The decisive checks are three. At $N\le300$, the constrained, symmetry-reduced estimator must beat projected Moment Unfolding and symmetrized linear inversion on moment error. It must hold nominal coverage where baselines undercover. It must keep the largest block solvable within a fixed single-GPU memory budget at degrees and particle counts where the unreduced program exceeds it. If only the third holds, the contribution is computational. If none holds beyond the symmetrization baseline, the result is a clean negative answer: under scarcity, symmetry helps the statistics but positivity does not.
References
- Borel measures with a density on a compact semi-algebraic set
- Symmetry groups, semidefinite programs, and sums of squares
- Optimal Transport with Symmetry Groups
- Matrix-Free GPU Semidefinite Programming for Quantum Ordered Search at the k=6 Frontier
- The matching problem has no small symmetric SDP
- OmniFold: A Method to Simultaneously Unfold All Observables
- Moment Unfolding
- Disentangling Quarks and Gluons with CMS Open Data