A model that learns tasks one after another has to balance two things: staying flexible enough to learn the new task and stable enough not to overwrite the old ones. A popular way to get stability is gradient projection. After each task you record which directions of weight change would disturb the old features, then block those directions for all later updates. The blocked set only grows, so the room left for learning shrinks until the model can no longer learn at all. Recent hyperbolic continual learners do the same thing. They place features in hyperbolic space, a curved geometry whose volume grows exponentially with distance from the center, so trees and hierarchies fit without crowding. But they choose the blocked directions with ordinary Euclidean linear algebra on the raw coordinates. This note argues that this choice is a measurable error, and that it wastes the capacity the method depends on. In the hyperboloid coordinates these methods use, the Euclidean length of a small move of a deep, specific concept can exceed its true hyperbolic length by a factor growing like $\cosh 2r$ in the concept's depth $r$. The protected basis therefore spends rank guarding motions that barely matter and underweights the ones that do.
We propose to choose protected directions by how much they would distort old embeddings as measured in the hyperbolic metric itself. The claim is concrete. At a fixed budget of hyperbolic distortion on old tasks, a metric-aware basis should need fewer protected dimensions than a Euclidean one. That should postpone the point where learning stops, and the benefit should grow with task count, with the depth spread of the concepts, and with curvature. The idea replaces a hard-coded rule used by the closest prior work, freezing the radial coordinate entirely, with a principled weighting that lets new tasks place concepts at new depths.
What exists and where the gap is
The mechanism we modify comes from Hyperbolic Multimodal Continual Learning (HMCL). HMCL works in the Lorentz model, where each point has a "time-like" coordinate $x_0$ encoding depth in the hierarchy and "space-like" coordinates encoding semantic direction. It forbids any change to the time-like part of the transformation layer. It also projects space-like gradients orthogonally to a low-rank SVD basis of past representations, and it reports a 74.5% relative reduction in forgetting over vanilla fine-tuning on a 15-dataset stream with a MERU-L backbone at fixed curvature $K=0.1$. Hierarchical Semantic Tree Anchoring for CLIP-Based Class-Incremental Learning (HASTEN) likewise projects gradients of a hyperbolic mapper into the null space of past features, on top of an LLM-generated taxonomy. Both methods motivate this work, and both leave the geometry out of the step that decides what to protect. We keep their overall recipe and change only how the protected basis is computed.
The failure we target is documented in Euclidean networks by Plasticity Activation via Polar Operator. That paper shows that gradient-projection methods such as Gradient Projection Memory (GPM) suffer "singular value collapse": as task subspaces are locked, the spectrum of the allowed update falls off a cliff. Its fix, PAPO, lifts every singular value of the projected gradient by a constant. This constrains our design, because any claimed gain in plasticity must be compared with PAPO applied directly to HMCL. Two related works, Restricted Orthogonal Gradient Projection and CODE-CL, relax strict orthogonality using task similarity. Both stay in flat space, and neither asks in which metric the damage to old features should be measured.
Other hyperbolic continual learners avoid projection altogether. HypCL composes task adapters with Möbius addition, the hyperbolic analogue of vector addition. Continual Hyperbolic Learning of Instances and Classes uses hyperbolic distillation toward fixed WordNet prototypes. We treat both as competitors rather than as starting points.
Core Claim
Consider a layer whose outputs are features $z_i$ on the hyperboloid. A small weight change $\Delta W$ moves old feature $i$ by $\delta_i = \Delta W u_i$, where $u_i$ is that sample's input to the layer. The first-order hyperbolic damage is the Lorentzian norm of $\delta_i$ after projecting it onto the tangent space at $z_i$. The Lorentzian norm is the length measured with the metric of signature $(-1,1,\dots,1)$, and the tangent space is the set of directions along the surface at that point. The quantity to protect is therefore
$$D(\Delta W) = \sum_i \big\| P_{z_i}\, \Delta W u_i \big\|_{\mathcal{L}}^2 ,$$
where $P_{z_i}$ is the tangent projection at $z_i$ and $\|\cdot\|_{\mathcal{L}}$ is the Lorentzian norm, which is positive on tangent vectors. Euclidean SVD instead implicitly protects $\sum_i \|\Delta W u_i\|_2^2$.
The two differ sharply with depth. For a point at hyperbolic radius $r$, a unit radial tangent move has squared Euclidean length $\cosh 2r$ in ambient coordinates but Lorentzian length exactly 1. The Euclidean energy threshold therefore spends protected rank on radial motion of deep, fine-grained concepts, whose true damage is small. Meanwhile, directions that matter for shallow, coarse concepts can fall below the threshold. HMCL's rule $\Delta x_0 = 0$ is the extreme response: it makes all radial motion infinitely costly, which also stops new tasks from learning where their own concepts sit in the hierarchy.
Our prediction follows directly from the $\cosh 2r$ factor. The mismatch between the Euclidean and metric-aware bases should grow with the spread of depths among old concepts and with curvature, since at fixed ambient norm the hyperbolic radius scales with $\sqrt{K}$. The mismatch should vanish in the flat limit.
Research Program
We will build the metric-aware protection operator in three steps.
The first step is exact but small scale. We form the quadratic form $D$ as a matrix over the vectorised weight change. We keep its top eigen-directions up to a hyperbolic-energy threshold, the analogue of GPM's 97% energy rule, and project new gradients onto the complement. This is feasible for the hyperbolic heads HMCL trains, and it serves as ground truth.
The second step makes the operator scale. We group old samples into shells by hyperbolic radius. Within each shell the output metric is nearly constant, so $D$ factorizes approximately into an input covariance times an output metric, a Kronecker product per shell. Protection then costs a few small eigendecompositions per task rather than one large one. We measure the factorization error against the exact version.
The third step drops the hard time-like freeze. Radial motion of old points is protected with its true weight, not infinite weight, so new tasks may set the depth of their own concepts.
Alongside the method we run a diagnostic study, which is itself a contribution. For Euclidean, log-map, and metric-aware bases, we track three curves over long streams: the effective rank of the admissible update space after each task, the hyperbolic distortion of old embeddings, and the preservation of radial ordering. Radial ordering means whether parents stay closer to the origin than their children. These curves tell us whether the spectral cliff PAPO found in Euclidean networks also appears in hyperbolic heads, and whether the choice of metric moves it.
Baselines
The naive reference point is sequential fine-tuning of the same hyperbolic head with no protection. It fixes the forgetting ceiling, and HMCL's reported 74.5% relative forgetting reduction is measured against exactly this reference. Alongside it we run Euclidean GPM on the same head, treating the hyperboloid coordinates as flat vectors. This shows what projection achieves when the geometry is ignored.
The strongest direct competitor is HMCL itself, with its zero time-like update and Euclidean SVD on space-like components. It is the published method closest to ours and the only one reporting results on the 15-dataset MERU stream. We run it from its described protocol on MERU-B and MERU-L at $K=0.1$. It gets the same frozen backbone, epochs, and optimizer as our method, and a threshold-search budget matched to ours, for example the same number of energy-threshold settings tried on a held-out task ordering. Beating it means lower backward-transfer forgetting than HMCL at equal or better average accuracy on the 15-dataset stream. We also require that the gap widens over later tasks rather than shrinking.
For class-incremental CIFAR-100, the strongest reported number in the file is HypCL's 89.09% average and 82.95% final accuracy in the B0 Inc10 protocol. We rerun HypCL in the same protocol. The same hyperbolic head on the same backbone is required for a fair comparison, so we will say explicitly if HypCL's CLIP adapters make the backbones incomparable. HASTEN serves as a second hyperbolic null-space competitor on CUB-200, Stanford-Cars, and ObjectNet. The file reports only its gains over the runner-up (2.18% and 2.34% final accuracy), not absolute numbers. For CUB-200 no reported result was found on Lacuna for any of the hyperbolic methods, so there we rely entirely on our reruns.
We run three ablations to isolate the key ingredient. The first keeps HMCL's hard time-like freeze and swaps only the space-like basis to the metric-aware one, which separates the value of metric weighting from the value of freeing radial motion. The second uses the log-map baseline described below. The third, a matched-rank control, forces both bases to protect the same number of dimensions and compares only the hyperbolic distortion they allow.
Why not just…?
Why not just add PAPO to HMCL? This is the most serious threat. PAPO lifts collapsed singular values and gave a +8.90% gain on GPM in the 20-split MiniImageNet setting reported by Plasticity Activation via Polar Operator. If HMCL+PAPO matches our method on the long streams, then choosing the metric carefully adds nothing beyond reviving plasticity. The two differ in kind, however. PAPO restores update magnitude inside a fixed blocked set and needs a hand-set strength $\lambda$. Our method changes which directions are blocked at all. We will run HMCL+PAPO and ours+PAPO. The decisive comparison is hyperbolic distortion of old tasks: PAPO should buy plasticity at some cost in distortion, while our method should buy it at none. If the combination beats both, that is the result we will report.
Why not just take the logarithmic map at the origin and run ordinary SVD there? This cheap fix, which sends each point to the flat tangent space at the center of hyperbolic space, removes the $\cosh 2r$ radial overweighting, because radial lengths are exact in those coordinates. It therefore may capture much of the gain. What it gets wrong is angular motion: at depth $r$ the true cost of a sideways move is larger by a factor of roughly $\sinh r / r$, so this basis underprotects fine-grained concepts against semantic drift. This alternative is a core ablation, not a footnote. If it matches the full metric on all benchmarks, we will say so and recommend it as the practical method, with the diagnostic study as the main contribution.
Why not just weight the projection by the Fisher information of the task loss? The Fisher information measures how sensitive the loss is to each parameter, so it already weights directions by what hurts old tasks. It does so, however, only through the old labels, and it vanishes on confidently classified samples. It therefore does not protect cross-modal retrieval geometry or radial hierarchy, which HMCL identifies as the failure modes of hyperbolic continual learning. We include a Fisher-weighted projection baseline. If it matches us on accuracy, we expect our method to win on retrieval recall and radial-order preservation; if it does not win there, the claim fails.
Why not just use Euclidean features with more dimensions? The evidence in the file already argues against this. Continual Hyperbolic Learning of Instances and Classes reports 41.76% versus 20.05% instance accuracy against iCaRL on EgoObjects, and HypCL beats the Euclidean adapter method Engine on CIFAR-100. Our question, though, is how to protect a hyperbolic model, not whether to use one. The Euclidean GPM baseline covers the flat case.
Proposed Experiments
The main testbed is HMCL's 15-dataset multimodal stream with MERU-B, MERU-L, and HyCoCLIP-B backbones. We report backward transfer, average accuracy, and cross-modal retrieval recall. To probe the long-stream regime where the spectral cliff appears, we split CIFAR-100 into 10, 20, and 50 tasks on the same frozen backbones. CUB-200 and the 15-task EgoObjects split with its 12-level WordNet taxonomy supply deep, fine-grained hierarchies. On these we additionally report Lowest Common Ancestor error distance, which measures how far apart in the class tree a wrong prediction lands, and radial-order preservation.
Four checks decide the claim. First, at matched protected rank, the metric-aware basis must give lower hyperbolic distortion of old embeddings than the Euclidean basis. At matched distortion, it must use fewer dimensions and keep a higher effective rank late in the stream. Second, its advantage over HMCL must grow with task count and be largest on EgoObjects and CUB, where depths spread widest. Third, in a curvature sweep from $K=0.01$ to $K=1$, the gap must shrink toward zero as curvature falls, as the $\cosh 2r$ argument predicts. Fourth, a controlled synthetic stream of WordNet subtrees with adjustable depth must show the gap tracking depth spread. If the gap does not track curvature and depth, the mechanism is wrong even if accuracy improves. If the log-map ablation closes the gap, the lesson is that choosing the right coordinates suffices and the full metric is unnecessary.
References
- Hyperbolic Multimodal Continual Learning
- Hierarchical Semantic Tree Anchoring for CLIP-Based Class-Incremental Learning
- Plasticity Activation via Polar Operator: A Plug-in Method for Balancing Stability and Plasticity
- Restricted orthogonal gradient projection for continual learning
- CODE-CL: Conceptor-Based Gradient Projection for Deep Continual Learning
- HypCL: Adapting CLIP in Hyperbolic Space for Continual Learning
- Continual Hyperbolic Learning of Instances and Classes