Low-Rank Spin Density Matrix Tomography for Rare B Decays

Rare decays of B mesons into a kaon resonance and a lepton pair are one of the main ways physicists look for particles too heavy to produce directly. These decays only happen through quantum loops, so a heavy unknown particle in the loop can shift the decay rate and the angles at which the final particles fly out. The difficulty is that the data are tiny: a few hundred to a few thousand events spread across many bins of lepton-pair mass and three decay angles. The usual remedy is to fit a full theory model. That model includes a poorly known Standard Model contribution from virtual charm quarks, and this contribution can imitate the very signal being searched for. We propose a different estimator. Before any theory model is assumed, the angular data in each mass bin can be read as measurements of a small complex matrix, the spin density matrix of the decay (a positive, Hermitian matrix that summarizes the quantum state of the final-state spins). Physics forces this matrix to have very low rank. Stacking these matrices across mass bins, lepton flavors (electron, muon, tau), and B-meson charge states gives an incomplete complex tensor, which is a multi-way array of numbers with many cells unobserved. Low-rank tensor completion can then pool the sparse data and test structured hypotheses directly.

The two-sentence version is this. Angular distributions of $B \to K^*\ell^+\ell^-$ are quantum state tomography of a rank-two density matrix, so the rank and positivity constraints that make compressed-sensing tomography work can also shrink the error bars on rare-decay angular observables. Arranging those matrices as a tensor over lepton flavor turns the question "is there lepton-flavor-dependent new physics, or only lepton-universal charm pollution?" into a question about rank along one tensor axis, and that question can be tested without committing to a model of the charm loops.

Core Claim

In the decay $B \to K^*(\to K\pi)\ell^+\ell^-$, the full four-fold angular distribution in $(q^2, \cos\theta_K, \cos\theta_l, \phi)$, where $q^2$ is the squared lepton-pair mass, is built from bilinears of a handful of complex helicity (transversity) amplitudes. For each lepton chirality $\chi \in \{L, R\}$ there is a complex three-vector $a_\chi(q^2) = (A_0, A_\parallel, A_\perp)_\chi$. In the massless-lepton limit, the measurable angular coefficients are linear functionals of

$$\rho(q^2) = \frac{a_L a_L^\dagger + a_R a_R^\dagger}{\operatorname{Tr}(a_L a_L^\dagger + a_R a_R^\dagger)},$$

a $3\times 3$ Hermitian, positive semidefinite, unit-trace matrix of rank at most two. Lepton-mass terms and scalar or time-like amplitudes enlarge the matrix and raise the allowed rank by a known amount. This matters most for the tau channel. Experiments currently fit the angular coefficients bin by bin as free trigonometric-polynomial weights. The research file notes that no existing analysis formulates the decay as a density operator (Angular distributions in the decay B → K^* l^+ l^- extracts $A_{FB}$ and $F_L$ by unbinned likelihood in two $q^2$ regions). The free per-bin fit ignores the positivity and rank constraints, so it spends statistical power on directions that are physically impossible.

The claim has two parts. First, maximum-likelihood estimation of $\rho(q^2)$ under a rank-$r$ positive semidefinite constraint gives tighter intervals on the angular coefficients at correct frequentist coverage than the unconstrained per-bin fit, and the gain is largest exactly where data are sparsest. Second, consider the tensor $\mathcal{T}[q^2\text{-bin}, \ell, \text{channel}]$, whose entries are these density matrices. Charm-loop contributions are lepton-universal: they enter only through the hadronic side. Under them, the lepton-flavor mode of the tensor admits a specific low-rank structure. That structure is a shared hadronic amplitude plus known kinematic mass corrections. Lepton-flavor-dependent short-distance new physics adds a component that this structure cannot absorb. A rank test along the flavor mode is therefore a model-agnostic discriminator between "anomalies caused by charm" and "anomalies caused by new physics," which is the central open problem raised in Lessons from the B^{0,+}\to K^{*0,+}\mu^+\mu^- angular analysis.

Relation to Prior Work

Prior work plays three roles here. Some of it supplies the method, some motivates the problem, and some sets constraints on the design.

On the method side, Quantum State Tomography via Compressed Sensing shows that a rank-$r$ density matrix of dimension $d$ can be recovered from $O(rd\log^2 d)$ expectation values by trace-norm minimization. We reuse the core idea that low rank lets a density matrix be estimated from far fewer effective measurements. Our measurements, however, are fixed angular moments with Poisson noise rather than chosen random Pauli operators, and our matrices are tiny ($3\times3$ up to roughly $6\times6$). The gain comes from pooling across bins and flavors, not from asymptotic dimension. Quantum state tomography with tensor train cross approximation stacks density matrices into a tensor network. Its result that per-basis sample requirements remain the real bottleneck (the "precision wall") is a warning we take seriously: rank constraints cannot create information that the event count does not contain. We address this by evaluating coverage, not only point error.

For the tensor side we draw on Tucker-format Riemannian completion (Implicit Regularization and Entrywise Convergence of Riemannian Optimization for Low Tucker-Rank Tensor Completion), whose entrywise guarantees fit our need to bound error on specific observables. We also draw on the autoregressive smoothing along an ordered mode in Low-Rank Autoregressive Tensor Completion for Spatiotemporal Traffic Data Imputation. There it handles block "blackouts" in time; here it handles the vetoed charmonium windows in $q^2$ around the $J/\psi$ and $\psi'$, which are structured, non-random missingness of exactly that kind. The file flags a known weakness of completion theory: incoherence assumptions fail when observed entries concentrate along physical peaks. Our setting has this property, so we cannot lean on generic recovery theorems and must validate by simulation.

From the quantum-inspired NLP literature we take parameterization tools, not semantics. End-to-End Quantum-like Language Models with Application to Question Answering builds density matrices as weighted sums of outer products of complex vectors and trains them end to end. Our $\rho = \sum_\chi a_\chi a_\chi^\dagger/\mathrm{Tr}$ is exactly that construction, and the file's noted Cholesky-style parameterization $\rho = LL^\dagger/\operatorname{Tr}(LL^\dagger)$ with $L \in \mathbb{C}^{d\times r}$ gives us positivity by construction. Quantum-Inspired Complex Word Embedding separates amplitude from phase in polar form. In our setting this split has a physical meaning: the shared strong phases come from long-distance hadronic effects, and weak phases from the couplings. We use polar factors in the flavor-shared hadronic component so that a lepton-universal strong phase is one parameter rather than many. The file raises a common objection to complex embeddings in NLP, that their gains may come only from doubling the parameter count. That objection does not apply here, because the amplitudes are complex by physics. The objection does constrain us in another way: the ablation against a real-valued, magnitude-only completion must be run at matched parameter count. These NLP papers do not touch physical data. We do not borrow their semantic claims, only their parameterization and optimization lessons, including the phase wrap-around problem at $\pm\pi$.

The motivating physics comes from Lessons from the B^{0,+}\to K^{*0,+}\mu^+\mu^- angular analysis. It fits LHCb angular data in the Standard Model Effective Field Theory (the standard parameterization of heavy new physics as coefficients of local operators) with a parameterized charm-loop term $h_\lambda$. It finds that a single left-handed operator improves the information criterion by $\Delta IC = 29$, about $6\sigma$ over the Standard Model, while stressing that hadronic power corrections can mimic a shift in the coefficient $C_9$. The tau channel, where flavor-mode tests would be most informative, is barely measured. Search for the decay B^0\to K^{*0}\tau^+\tau^- at the Belle experiment sets only an upper limit, $\mathcal{B} < 3.1\times10^{-3}$ at 90% confidence level. That extreme missingness is part of the motivation for completion.

Research Program

The first component is the measurement map. For each $q^2$ bin, lepton flavor, and channel, we write the angular distribution as $\operatorname{Tr}(\rho\, M(\Omega))$, where $\Omega$ denotes the three angles and $M(\Omega)$ is a known Hermitian operator-valued function built from Wigner functions. Each event is then a sample from a quantum measurement on $\rho$. We will verify that this reproduces the standard angular coefficients, including $A_{FB}$, $F_L$, and $P_5'$. We will derive the extended matrix and its rank bound when lepton masses and scalar amplitudes are kept, and we will check positivity of $\sum_\Omega M(\Omega)$-type completeness so that the measurement is a valid one. The file lists measurement-operator completeness as an open issue in the NLP version of this construction; here it is a computable fact.

The second component is the estimator. We will build an unbinned maximum-likelihood fit over $\rho = LL^\dagger/\operatorname{Tr}(LL^\dagger)$ with complex $L$ of rank $r$. Bins will be coupled through a Tucker decomposition of the stacked tensor whose flavor-mode factor is split into a polar-parameterized universal hadronic part and an optional flavor-specific part. An autoregressive penalty along $q^2$ fills the charmonium vetoes. Confidence intervals will come from profile likelihood with parametric bootstrap, because rank constraints make boundary effects severe and naive Hessian errors would be wrong.

The third component is the flavor-rank test. The null hypothesis is that the flavor-mode structure equals universal plus known mass corrections. The alternative adds one free flavor-specific component. The test statistic is the likelihood ratio, and its null distribution comes from pseudo-experiments, since boundary conditions void Wilks' theorem. We will map its power against injected lepton-flavor-dependent shifts and its false-positive rate against injected universal charm effects of varying size and $q^2$ shape.

Baselines

The naive reference is the per-bin unconstrained angular fit. This is the method used in Angular distributions in the decay B → K^* l^+ l^-. At the BABAR scale of 384 million $B\bar B$ pairs it reports, at low $q^2$, $A_{FB} = 0.24^{+0.18}_{-0.23}\pm0.05$ and $F_L = 0.35\pm0.16\pm0.04$. At high $q^2$ it reports $A_{FB} = 0.76^{+0.52}_{-0.32}\pm0.07$ and $F_L = 0.71^{+0.20}_{-0.22}\pm0.04$. We will rerun this fit on the identical pseudo-datasets, with the same efficiency model and the same event counts. Beating it means a statistically significant reduction in interval width at no loss of coverage, measured as the fraction of pseudo-experiments whose interval contains the truth.

The strongest existing competitor is the full effective-theory global fit with parameterized hadronic corrections, as performed with HEPfit in Lessons from the B^{0,+}\to K^{*0,+}\mu^+\mu^- angular analysis. Its reported strength of evidence is $\Delta IC = 29$, about $6\sigma$, for one left-handed operator. It is strongest because it uses all theory knowledge. We will compare on pseudo-data generated from charm-loop shapes both inside and outside its assumed $h_\lambda$ parameterization, with equal data and equal numbers of free hadronic parameters. Beating it does not mean higher significance on well-specified data, where it should win. It means a lower false new-physics rate when the true charm shape lies outside its parameterization, at comparable power when it lies inside.

The key-ingredient ablations isolate what each constraint contributes. The first drops positivity (low-rank Hermitian but not positive semidefinite). The second drops cross-bin coupling (per-bin rank-two density matrices only). The third replaces complex factors with real factors at matched real parameter count, which addresses the parameter-doubling critique directly. As a further generic baseline we will run off-the-shelf real-valued Tucker and tensor-train completion (Efficient tensor completion: Low-rank tensor train) on the binned angular-moment tensor itself. All methods get the same hyperparameter search budget, selected on a held-out set of pseudo-experiments.

Why not just…?

Why not just use the global effective-theory fit, which already handles charm loops? It handles them through an assumed functional form. When that form is right, the global fit is the best possible analysis, and we expect it to beat us on power. The open problem named in the Lessons paper is precisely that the form might be wrong. Experiment 2 settles this: if the global fit's false-positive rate stays at its nominal level even under mis-specified charm shapes, then our flavor-rank test adds little and we will report that.

Why not just smooth the per-bin coefficients across $q^2$ with splines or a z-expansion? Smoothing pools information along $q^2$ but does not enforce positivity or rank, so it still allows impossible coefficient combinations. It also says nothing about flavor structure. This alternative is a real threat on the single-flavor muon data. The "drop positivity, keep coupling" ablation measures exactly how much of our gain is smoothing alone. If that share approaches the full gain, the density-matrix framing is not earning its place for the first claim, and the paper's weight shifts to the flavor-rank test.

Why not just apply standard tensor completion to binned event counts? Counts in sparse four-dimensional bins are mostly zero. Completion theory needs incoherence, which fails at physical peaks, and real-valued factorizations cannot represent interference phases. The generic tensor-train and Tucker baseline tests this directly. We expect it to produce unphysical angular distributions, which we will measure as the fraction of negative reconstructed densities.

Why not just train a neural density estimator on the events? With hundreds of events a flexible model has no defensible coverage, and its outputs do not separate a universal component from a flavor-specific one. We regard this as weaker and will include it only as a coverage check.

Proposed Experiments

Experiment 1 tests estimation efficiency. We generate pseudo-experiments from Standard Model amplitudes at BABAR scale (384 million $B\bar B$) and at Belle scale (772 million $B\bar B$, 711 fb$^{-1}$), with realistic acceptance and the charmonium vetoes. We compare interval width and coverage for $A_{FB}$, $F_L$, and $P_5'$ across all baselines. The decisive check is whether the rank-constrained estimator narrows intervals at nominal coverage. The second check is whether, applied to the published BABAR selection, it gives results compatible with the quoted BABAR values above.

Experiment 2 tests charm versus new physics. We inject lepton-universal charm terms with shapes inside and outside the HEPfit parameterization, and separately inject lepton-flavor-dependent shifts to $C_9$ and $C_{10}$. For each scenario we measure the false-positive and power curves of the flavor-rank test against the global fit. The proposal succeeds if the rank test keeps nominal false-positive rate under all universal injections while the global fit's rate rises under mis-specified shapes.

Experiment 3 tests tau prediction. We treat the tau slice as almost entirely missing, consistent with only an upper limit of $3.1\times10^{-3}$ being available, and use the completed tensor to predict tau angular coefficients under each hypothesis. We then check how strongly a future Belle-scale tau measurement could discriminate between the two hypotheses. No reported tau angular result was found on Lacuna, so this experiment uses pseudo-data only.

Experiment 4 is the application. If Experiments 1 and 2 pass, we apply the method to the publicly available binned LHCb muon results used in the Lessons paper, together with electron-channel inputs. We report the flavor-rank statistic alongside, not instead of, the global-fit significance.