Smoothness Penalties That Leave Smooth Drift Alone

Models that learn one embedding per time step, such as dynamic graph embeddings, temporal knowledge graph factorizations, and diachronic word vectors, almost always add a penalty on the change between consecutive steps, $\|u_t - u_{t-1}\|^2$. That penalty works in practice, but it has a built-in bias. It charges a node that drifts steadily across a hierarchy the same as a node that jitters randomly, and it charges a sharp but genuine change heavily. Geometric modeling met the same problem decades ago and solved it with interpolatory subdivision. A subdivision scheme is a rule that predicts each in-between point from its neighbors. A good scheme predicts any cubic trajectory exactly, so the leftover "detail", meaning actual point minus predicted point, is zero on smooth motion and large only on real irregularities.

We propose to use these multiscale details, computed intrinsically on the embedding manifold, as the temporal regularizer. Proximity analysis is the body of theory showing that a curved-space version of a linear scheme inherits the linear scheme's accuracy when the data are dense. It tells us when this penalty keeps its key property on hyperbolic or spherical embeddings and when a cheaper version breaks. The same subdivision rule then answers queries at timestamps never seen in training. Lacroix et al. and Yao et al. both leave that capability open.

Core Claim

The claim is that a temporal regularizer should vanish, to high order in the sampling interval, on smooth trajectories of the embedding manifold, and should penalize only what those trajectories cannot explain. Write the embeddings of one entity at dyadic levels as $z_{\ell,i} \in M$, where $M$ is the embedding manifold (for example the Poincaré ball). Let $P$ be the intrinsic version of the Dyn–Levin–Gregory four-point rule. It is built by log–exp averaging: map the four even-indexed neighbors into the tangent space at $z_{\ell,2i}$, take the weights $(-1,9,9,-1)/16$, and map back. The detail and the penalty are then

$$d_{\ell,i} = \log_{P(z_\ell)_{2i+1}}\big(z_{\ell,2i+1}\big), \qquad R(z) = \sum_{\ell} w_\ell \sum_i \rho\big(\|d_{\ell,i}\|\big),$$

where $w_\ell$ weights each scale and $\rho$ is either squared or Huber. The Huber choice lets a few large details, which are genuine transitions, survive while many small ones are suppressed. That is the wavelet-shrinkage logic, and it addresses the oversmoothing trade-off that Knights et al. observed.

The linear four-point rule reproduces cubics, so the Euclidean detail on samples of a smooth curve with spacing $h$ scales like $h^4$ by a Taylor remainder argument. A first-difference penalty scales like $h$. The proximity results imply that the intrinsic version keeps the high-order decay. Approximation order of interpolatory nonlinear subdivision schemes proves that a nonlinear analogue satisfying proximity inequalities inherits the linear scheme's approximation order. Smoothness Analysis of Subdivision Schemes on Regular Grids by Proximity covers log–exp and geodesic analogues explicitly. The guaranteed order in the nonlinear case is governed by the scheme's smoothness, which is $h^{r+1}$ for a $C^r$ scheme per Dyn et al., and may be lower than the linear $h^4$. The empirical slope is therefore something we measure, not something we assume.

Relation to Prior Work

Four papers motivate the proposal by exhibiting the gap. Tensor Decompositions for Temporal Knowledge Base Completion adds $\|w_{t+1}-w_t\|^2$ to an order-4 complex factorization and gains up to 0.07 MRR. It explicitly cannot handle sub-bucket times. Dynamic Word Embeddings for Evolving Semantic Discovery uses a Frobenius first-difference penalty in a joint factorization of word co-occurrence statistics across years, and names continuous time as open. Discrete-time Temporal Network Embedding via Implicit Hierarchical Learning in Hyperbolic Space (HTGN) penalizes squared Poincaré distance between consecutive snapshots, and ablating that term costs up to 19%. Temporal Subspace Clustering for Human Motion Segmentation applies a first-order temporal Laplacian to subspace coefficients. All four use one-step, single-scale penalties with no notion of which trajectories should cost nothing. We replace the penalty in each and keep the rest of the model fixed.

Temporally Coherent Embeddings for Self-Supervised Video Representation Learning constrains the design. It shows that pushing trajectories toward straightness hurts fast actions. We read this as evidence against penalties that annihilate only straight lines and penalize every frame uniformly. Our penalty differs on both counts: it annihilates cubics, and it tolerates sparse large details. If it still oversmooths, that finding counts against us.

The subdivision papers supply the method and its limits. A General Proximity Analysis of Nonlinear Subdivision Schemes gives global convergence on non-positively curved spaces, which include hyperbolic space, but only for non-negative masks. The four-point mask has negative weights, so we inherit only the local, dense-data guarantee. We therefore monitor the spacing $\Delta$ between consecutive embeddings throughout training. Definability and stability of multiscale decompositions for manifold-valued data proves that non-interpolatory manifold filters fail a rank condition needed for perfect reconstruction. This is why we use an interpolatory pyramid (keep the even points, compute details at the odd points) and not a B-spline-type smoother.

Research Program

We first build the intrinsic detail pyramid as a differentiable layer for the Poincaré ball, for the sphere, and for flat space. The flat case reduces to a fixed linear filter. We also build a "tangent-at-origin" variant that computes the four-point rule in the chart at 0, which is how HTGN's recurrent unit handles geometry. Proximity theory predicts that this variant's error relative to the intrinsic one grows with distance from the origin, where the chart distorts.

Second, we derive and verify scaling laws on synthetic geodesic-polynomial curves. For each penalty we plot penalty size against $h$, stratified by curvature and by distance from the origin.

Third, we insert $R$ into HTGN, TComplEx, the dynamic word embedding model, and temporal subspace clustering, in each case replacing only the temporal term.

Fourth, we use the subdivision rule as a decoder. Held-out intermediate snapshots or timestamps are predicted by refining the learned neighboring embeddings. This tests whether a regularizer that is consistent with the scheme yields trajectories the scheme can interpolate.

Baselines

The do-nothing reference is each host model with its temporal term removed. For HTGN, the ablation of its consistency term in the cited paper reports drops of up to 19%. We rerun this ourselves rather than quote it.

The strongest existing method on the hyperbolic question is HTGN with its first-order hyperbolic consistency loss. We run it on HepPh, COLAB and Facebook with the authors' architecture, dimension and training budget. The temporal-term weight is tuned on the same validation grid used for our weights. For the HTGN baseline on these datasets, no reported result was found on Lacuna for per-dataset AUC; the file reports only relative gains of up to 9.98% (link prediction) and 11.4% (new link prediction). Beating it means a higher mean AUC over five seeds than our reproduced HTGN, with non-overlapping confidence intervals, on at least two of three datasets.

On temporal knowledge graphs, the baseline is TComplEx with its smoothness regularizer on ICEWS14 and ICEWS15-05, given identical rank and tuning budget. The paper reports up to +0.07 MRR from smoothness, but no reported result was found on Lacuna for its exact MRR. On motion segmentation, the baseline is temporal subspace clustering, which reports 57.12% accuracy on Keck Gesture, 76.15% on Weizmann and 47.67% on MAD. Beating it means exceeding these figures with the Laplacian swapped for $R$ and everything else unchanged.

Two ablations isolate the ingredients. The first is a single-scale Euclidean higher-order difference penalty, which tests whether multiscale structure matters. The second is the tangent-at-origin pyramid, which tests whether intrinsic computation matters.

Why not just…?

Why not just use a second- or third-order finite-difference penalty? In flat embedding spaces this is a real threat. A single-scale difference penalty of matching order also annihilates low-degree polynomials, so on TComplEx and the word-embedding model it may match us. We include it with an equal tuning budget and will report a tie honestly. What it lacks is the per-scale weighting with Huber details, which is what is supposed to protect sharp transitions. The Keck, Weizmann and MAD segmentation experiments, where action boundaries are exactly such transitions, are designed to separate the two. On curved embeddings, a Euclidean difference of manifold points is not even an intrinsic quantity.

Why not just tune HTGN's first-order loss and use geodesic interpolation at test time? Piecewise-geodesic interpolation is only $C^0$, and the first-order penalty charges steady drift. The held-out-snapshot experiment compares exactly this pairing against our regularizer plus subdivision decoding. If geodesic midpoints recover held-out snapshots as well as subdivision does, the decoding half of our claim fails.

Why not compute everything in the tangent space at the origin, which is cheaper? HTGN already does this for its recurrent unit. The cited paper notes distortion far from the origin, and proximity bounds hold only within charts where data are close. Our norm-stratified synthetic test and norm-stratified link-prediction errors will show whether the cheap variant degrades for peripheral (leaf-like) nodes. If it does not, we will adopt it.

Why not use a recurrent or continuous-time network? Lacroix et al. found that factorization with temporal priors beat recurrent alternatives in both scalability and accuracy, so we stay within regularized factorization. We do not claim to beat recurrent models.

Decisive Experiments

The first check is synthetic and cheap. On geodesic-polynomial curves in the Poincaré ball, the intrinsic detail penalty should show a log–log slope against $h$ that is clearly steeper than the first-difference penalty's slope of about 1. The tangent-at-origin variant's slope should flatten as curves move toward the boundary. If the intrinsic slope is not steeper, the core claim is false.

The second check holds out every other snapshot in HTGN's datasets and every other timestamp in ICEWS14. We train on the remaining ones and score link prediction and MRR at the held-out times. The comparison is between subdivision-decoded embeddings and nearest-snapshot or geodesic-midpoint embeddings under the first-order penalty.

The third check swaps the regularizer in the four host models under matched budgets. We report AUC, MRR, clustering accuracy against the TSC numbers above, and Total Absolute Curvature, the trajectory-smoothness measure from Knights et al., as a diagnostic. We also report the measured spacing $\Delta$ between consecutive embeddings, so readers can see whether the dense-data regime the theory needs actually held.