When several similarity measures describe the same data, for example a protein-language-model embedding and a 3D-structure similarity, or a bag-of-words view and a parse-tree view of a sentence, the usual practice is to add them with one global set of weights. That practice assumes each view is equally trustworthy everywhere in the data. It rarely is. We propose to let the weights vary from place to place and to set them without labels, using a geometric test. The test comes from manifold learning. Cut the data into small overlapping neighborhoods, give each one a flat local coordinate system (a "chart"), and ask whether those charts can be stitched into one consistent global map. A mix of kernels that makes neighboring charts agree on their shared points is picking up the data's real local geometry. A mix that makes them disagree is picking up noise.
The contribution is to use the error left over from stitching the charts as the objective for per-neighborhood kernel weights. The idea is to optimize those weights jointly with the global coordinates, rather than fixing the kernel first and aligning afterwards. The resulting field of weights can be used directly as an unsupervised embedding for clustering. It can also be turned into a valid, locally adaptive composite kernel for Gaussian-process regression, which should help most when labels are scarce. In that regime, fitting many local weights to labels would overfit, but unlabeled points still carry geometric information.
Core Claim
Global coordinate alignment methods, starting with Local Tangent Space Alignment (LTSA), solve one fixed problem. Each point's neighborhood gets local coordinates from a local principal component analysis. A single eigenproblem then finds global coordinates that are, in every neighborhood, as close as possible to an affine image of that neighborhood's local coordinates. The fitting residual is the part of the global map that no chart can explain. LTSA treats it as a quantity to minimize over coordinates only.
Our claim is that this residual is also a good criterion for choosing the kernel. The local principal component analysis needs only a centered local Gram matrix, so LTSA runs unchanged on any kernel, including tree, graph or composite kernels on non-vectorial inputs. The residual therefore becomes a function of the kernel weights.
Concretely, let $K_1,\dots,K_M$ be trace-normalized base kernels. Let $\mathcal{N}_i$ be the $k$ neighbors of point $i$, fixed in advance from the uniform-sum kernel so that the weights cannot pick their own neighborhoods. Let $\beta_i$ be a weight vector on the probability simplex for chart $i$. Let $P_i(\beta_i)$ be the projector onto the top-$d$ eigenvectors of the centered local Gram matrix $H\big(\sum_m \beta_{im} K_m[\mathcal{N}_i]\big)H$, where $H$ is the centering matrix. Let $Y_i$ be the global $d$-dimensional coordinates $Y$ restricted to $\mathcal{N}_i$. We solve
$$ \min_{Y YY^\top = I,\ \{\beta_i\}} \ \sum_i \big\| Y_i (H - P_i(\beta_i)) \big\|_F^2 \;+\; \lambda \sum_{i \sim j} \|\beta_i - \beta_j\|^2 , $$
where $i \sim j$ ranges over charts whose neighborhoods overlap. The second term makes overlapping charts negotiate their weights instead of choosing them independently.
With $\beta$ fixed, the problem is exactly the LTSA eigenproblem. With $Y$ fixed, each chart's step is equivalent to maximizing $\mathrm{tr}(P_i(\beta_i) Y_i^\top Y_i)$, since $\|Y_i(H-P_i)\|_F^2 = \|Y_i H\|_F^2 - \mathrm{tr}(P_i Y_i^\top Y_i)$ and the first term does not depend on $\beta_i$. That quantity is how much of the current global map's local variation the chart's top-$d$ subspace captures.
For downstream regression we use the learned weight field $\beta(x)$ to form
$$ K_\beta(x,x') = \sum_{m=1}^M \sqrt{\beta_m(x)\,\beta_m(x')}\;K_m(x,x'). $$
Each term has the form $g(x)g(x')K_m(x,x')$, so each term is positive semi-definite, and so is the sum. This gives a valid kernel whose mixture changes smoothly across the dataset.
Relation to Prior Work
The weight step deserves an honest statement of how close it is to existing work. In linearized form it is a local kernel-target alignment. Multiple Kernel Clustering with Local Kernel Alignment Maximization aligns a combined kernel with an ideal cluster-similarity matrix only within each point's nearest neighbors. That paper supplies our motivation, because it shows that discarding distant pairs lifts accuracy on Digital from 47% to 96.25%. It learns one global weight vector against discrete cluster indicators. We learn one weight vector per chart, against continuous global coordinates that are themselves re-solved from those charts.
Two other clustering methods also build local structure into multiple kernel learning. Manifold Adaptive Multiple Kernel K-Means deforms each base kernel with a neighbor-graph Laplacian. Multiple Kernel Clustering via Local Regression Integration fuses sparse local Nadaraya–Watson regression matrices. Both use a single global weight vector, and neither builds charts or resolves the affine ambiguity between them. Together these three papers are the strongest evidence that locality matters in kernel fusion. They constrain our design in one way: they show that locality in which pairs count already brings large gains. Our experiments must therefore separate that effect from locality in the weights themselves.
On the alignment side, Charting a Manifold and Global Coordination of Local Linear Models formalize the stitching of charts that are each defined only up to an affine transform. We reuse their closed-form, no-local-minima coordinate step. Local Deep-Feature Alignment replaces local principal component analysis with per-neighborhood auto-encoders but keeps a single fixed representation, and Local2Global stitches graph-patch embeddings by synchronizing their rotations, shifts and scales. Neither changes what the local model sees in response to how well it stitches, which is our core move.
Kernel Manifold Alignment is the closest kernelized alignment method. It maps separate domains into a shared space using label correspondences and fixed per-domain kernels. It does not build an internal atlas, and it does not learn kernel mixtures.
For regression, Kermut is the composite-kernel reference. It sums a sequence kernel built on ESM-2 embeddings and a structure kernel built on ProteinMPNN outputs, with hyperparameters set by marginal likelihood. Its weights are global.
Research Program
The first stage builds the solver. It alternates the LTSA eigenproblem, solved with sparse eigensolvers on the $N\times N$ alignment matrix, with per-chart weight updates. For the weight updates we will compare two options: exact projected gradients through the local eigenvectors, using standard eigenvector perturbation formulas with $M\le 12$ and $k$ in the tens, and the cheaper surrogate that maximizes alignment between the normalized local Gram matrix and $Y_i^\top Y_i$.
Degenerate solutions need explicit guards. Kernels are trace-normalized within each neighborhood. Neighborhoods are fixed in advance. We will also report the local explained-variance fraction of each chart, so that a kernel which "wins" only because its local Gram matrix happens to be nearly rank $d$ becomes visible. Out-of-sample points receive weights by kernel-weighted interpolation of their neighbors' $\beta$. In the ProteinGym setting we instead work transductively over the full enumerated variant set, which is known in advance.
The second stage is a controlled synthetic study with a known answer. We build a Swiss roll observed through two views. Each view is corrupted with high-dimensional noise over a different, known half of the roll, and a third view is pure noise. We then check whether the recovered weight field matches the true reliability map, and whether the stitched coordinates recover the roll better than uniform or globally weighted kernels do.
The third stage applies the embedding to unsupervised multi-kernel clustering. The fourth uses $K_\beta$ inside a Gaussian process for protein fitness prediction with few labels, where a label-free criterion has a principled advantage.
Baselines
The naive reference point is kernel LTSA on the uniform sum $\sum_m K_m$, followed by k-means or Gaussian-process regression. It shares our neighborhoods, embedding dimension and solver, and differs only in that nothing is learned. If the learned weight field does not beat it, the idea has failed.
The main ablation keeps our objective but forces a single shared $\beta$ for all charts. This isolates whether spatial variation in the weights, rather than chart-consistency weighting as such, is what helps. A second ablation sets $\lambda=0$, so charts pick weights independently, to test whether coupling across overlaps matters. A third replaces the chart-consistency weight step with local kernel-target alignment against k-means indicators. This is essentially the MKKM-LKAM criterion applied per chart, and it tests whether continuous self-consistent targets matter.
For clustering, the strongest existing competitors are MKKM-LKAM, MAMKKC and CMKLR. They are strongest because they already exploit locality, which is precisely what a skeptic will say explains any gain we report. We will rerun all three on the eight benchmarks and 12 candidate kernels used by MAMKKC (BBC, RELATHE, BASEHOCK, PIE10P, UMIST, COIL20, Leukemia, ALLAML), plus Digital. Every method gets the same kernels, the same neighborhood-size grid and the same number of restarts.
The published reference points are MAMKKC's reported average accuracy of 81.68% on those eight datasets and CMKLR's reported average accuracy of 0.8311 and NMI of 0.5212 on its own five benchmarks. Beating these baselines means higher average accuracy and NMI than the best rerun competitor under matched tuning, with the gain concentrated on datasets where the recovered weights actually vary across charts.
For regression, the strongest baseline is Kermut with global weights learned by marginal likelihood. It reports an average Spearman correlation of 0.662 over the 217 ProteinGym substitution assays under the random split, against 0.613 for ProteinNPT. We will rerun Kermut's kernels unchanged and swap only the global sum for $K_\beta$, with identical Gaussian-process hyperparameter fitting. No reported result was found on Lacuna for Kermut with subsampled training labels, so we will produce those numbers ourselves. Beating Kermut means a higher mean Spearman correlation at 50, 100 and 250 labeled variants per assay, together with no significant loss when all labels are used.
Why not just…?
Why not just run kernel LTSA on the sum of kernels? LTSA needs only local Gram matrices, so this is easy and is our naive baseline. Its weakness is that a view which is noisy in one region still receives full weight there. The synthetic study directly measures how much stitching error that causes, and the uniform-sum baseline on real data tells us whether the effect matters in practice.
Why not just learn global weights, as Kermut does by marginal likelihood or CMKLR does by local regression? This is the most serious competitor. If the relative reliability of the views is roughly constant across the data, a single weight vector is sufficient and our shared-$\beta$ ablation will match the full method. We treat that outcome as the decisive negative result rather than something to argue away. The locality results of MKKM-LKAM show that which pairs count matters, but not that weights must vary. Our experiments test the second claim separately.
Why not just train an input-dependent gating network for the weights on labels? With many labels this may well win, and the full-label ProteinGym runs will show whether it does. Our claim is limited to the unsupervised and low-label regimes. There a gating network has many parameters and few targets, whereas our criterion draws on every unlabeled variant. The gating network will be run at every label budget with the same kernels, so the crossover point will be reported rather than assumed.
Why not just concatenate embeddings and use an auto-encoder or UMAP? Tree, graph and structure-environment kernels have no tractable explicit feature vectors, which is why composite tree kernels exist at all. LDFA further reports that global auto-encoders underperform local alignment. For vector-only benchmarks we will still include a concatenate-then-LDFA baseline, so the comparison is not limited to the cases where it is impossible.
Decisive Checks
Three results would settle the question. The first is on synthetic data: the recovered weight field should correlate with the known per-region reliability of each view, and the pure-noise view should receive near-zero weight everywhere. If this fails, the criterion is not measuring what we claim. The second is on the clustering benchmarks: the full method should beat both the shared-$\beta$ ablation and the best rerun local multiple-kernel-learning method. The third is on ProteinGym, where the margin over Kermut should shrink as labels grow. A margin that is largest at 50 labels and near zero at full data would confirm that geometry substitutes for labels. A uniform gain at all budgets would instead suggest that the benefit comes from locality alone. We will also report chart explained-variance fractions and runtime against the $O(N^3)$ Gram cost that Kermut itself identifies beyond 10,000 variants.
References
- Nonlinear Dimension Reduction via Local Tangent Space Alignment
- Multiple Kernel Clustering with Local Kernel Alignment Maximization
- Manifold Adaptive Multiple Kernel K-Means for Clustering
- Multiple Kernel Clustering via Local Regression Integration
- Charting a Manifold
- Global Coordination of Local Linear Models
- Local Deep-Feature Alignment for Unsupervised Dimension Reduction
- Local2Global: a distributed approach for scaling representation learning on graphs
- Kernel Manifold Alignment
- Kermut: Composite kernel regression for protein variant effects
- Exploiting Tree Kernels for High Performance Chemical Induced Disease Relation Extraction