Exponent-Coded Hyperbolic Embeddings: Exact Distances from Shifts and Lookups

Hyperbolic embeddings place tree-like data, such as taxonomies, citation graphs and knowledge graphs, into a curved space where room grows exponentially with distance from the center. That is how a tree grows, so a 5-dimensional hyperbolic embedding can represent WordNet better than a 200-dimensional flat one. The cost is arithmetic. Every distance needs square roots, divisions, and inverse hyperbolic functions evaluated in 32-bit floating point, which is a poor fit for chips that lack floating-point units. Power-of-two quantization, which restricts numbers to $2^n$ so that multiplication becomes a bit shift, is the standard way to make networks run without multipliers. The two ideas look incompatible at first. Power-of-two grids are dense near zero and sparse far from it. Hyperbolic embeddings put their most important points, the leaves, in a thin shell near the boundary of the ball, which is exactly where such grids are coarsest.

This proposal argues that the incompatibility comes from the choice of coordinates, not from the geometry. We store each point as a unit direction plus the base-2 logarithm of a hyperbolic function of its radius. In that representation, hyperbolic distance can be computed exactly as a sum of two positive terms. Everything then reduces to integer additions, a few small lookup tables, and log-domain addition, which is the arithmetic of logarithmic number systems (a generalization of power-of-two coding with fractional exponents). A logarithmic grid in this coordinate is uniform in true hyperbolic distance. As a result, the bits needed to store depth should grow with the logarithm of tree depth rather than linearly with it. The expected outcome is multiplierless hyperbolic distance evaluation that loses almost nothing relative to FP32 on hierarchy benchmarks, where naive quantization of the standard coordinates fails.

Core Claim

Both standard hyperbolic coordinate systems break under low-bit numerics, and they break for opposite reasons.

The Poincaré ball model is a bounded unit ball in which distances blow up near the edge. In this model, a point at hyperbolic radius $r$ sits at Euclidean norm $\tanh(r/2)$. Its gap to the boundary shrinks like $2e^{-r}$. Any fixed-point or power-of-two code on the coordinates therefore needs roughly $r/\ln 2$ bits just to tell depth levels apart near the rim. This is the density clash flagged by the research file, contrasting FQ-ViT, whose log2 grids work because attention values cluster near zero, with Poincaré Embeddings, whose expressiveness depends on points pushed outward.

The Lorentz model is a hyperboloid in Minkowski space, adopted by Learning Continuous Hierarchies in the Lorentz Model precisely to avoid Poincaré boundary instability. Its coordinates grow like $e^{r}$, and a power-of-two code handles that magnitude easily. But its distance is $\operatorname{arcosh}$ of a Minkowski inner product that subtracts two huge, nearly equal numbers. For a parent and child deep in the tree, both terms are about $e^{2r}$ while their difference is order one. Any relative rounding error then wipes out exactly the short distances that encode the hierarchy.

The fix is an exact identity. Write a point $u$ as a unit direction $a_u \in \mathbb{R}^n$ and a hyperbolic radius $r_u$, and let $d(u,v)$ be the hyperbolic distance between $u$ and $v$. The hyperbolic law of cosines rearranges to

$$\cosh d(u,v) = \cosh(r_u - r_v) + \tfrac{1}{2}\,\sinh r_u \,\sinh r_v\,\|a_u - a_v\|^2 .$$

Both terms on the right are non-negative, so there is no cancellation anywhere. We store $\rho_u = \log_2 \sinh r_u$ as a fixed-point number with $f$ fractional bits, which is a base-$2^{2^{-f}}$ logarithmic quantizer. With $f=1$ this is the base-$\sqrt{2}$ grid used by LogART. We store each direction as low-bit integer coordinates.

The logarithm of the second term is then $\rho_u + \rho_v + \log_2\|a_u-a_v\|^2 - 1$. The squared direction difference is a sum of squared small-integer differences, obtained from a tiny squaring table. Its logarithm comes from a leading-one detector plus a lookup. The first term depends only on $\rho_u - \rho_v$ through a one-dimensional table. The two terms are combined by log-domain addition: take the larger, then add a correction from a table indexed by their difference. The final $\operatorname{arcosh}$ is nearly linear in the log domain for large arguments and handled by a table for small ones.

Rounding $\rho$ by $\delta$ shifts the log of the dominant term by at most about $2\delta$. That changes distance by an additive amount of order $\delta \ln 2$, independent of depth. This additive, depth-independent error is the property neither standard coordinate system has.

Relation to Prior Work

Three hyperbolic papers use the word "quantization," and all of them mean clustering vectors into a codebook: HyperVQ (codebook selection as hyperbolic logistic regression), Hyperbolic Residual Quantization (residual codebooks via Möbius subtraction), and HiHPQ (product quantization over Lorentz subspaces). Every code vector and distance in these methods remains FP32. They motivate our work by showing that discrete hyperbolic representations are useful, but they do not touch arithmetic precision. Our method is complementary, because a codebook entry could itself be exponent-coded.

Fast and Geometrically Grounded Lorentz Neural Networks is the closest speed-oriented work. It caches transport parameters and cancels transcendental functions, reaching near-Euclidean latency, but it stays in FP32 on GPUs. We target hardware with no floating-point unit at all.

On the power-of-two side, we reuse existing techniques in specific ways. LogART supplies learnable rounding on logarithmic grids and a shift-add realization of base-$\sqrt{2}$ values; we apply its rounding idea to $\rho$ but optimize a distance-distortion loss instead of a layer-output loss. SwiftTron approximates Euclidean nonlinearities with integer polynomials. Sorbet rounds normalization divisors to powers of two. Neither provides a hyperbolic distance. Unlike these approximations, ours is exact up to table resolution.

Bin Regularization constrains our design. Its 2-bit results show that low-bit grids need training-time help, so we plan quantization-aware training rather than relying only on rounding after training.

Research Program

The first component is the representation and its error analysis. We will derive worst-case and average distance error as a function of the radius bits and fractional bits $f$, the direction bit width $b$, and the table sizes. The analysis should confirm the depth-independent additive bound for the radial part. It should also identify the real bottleneck, which we expect to be angular: siblings deep in a tree differ mainly in direction. The number of distinguishable directions is at most $2^{b(n-1)}$, so direction bits, not radius bits, should set capacity.

The second component is training. We will train embeddings with Riemannian stochastic gradient descent, as in Poincaré Embeddings, in polar form. The radius is parametrized directly by $\rho$ and the direction lives on the sphere. We will apply quantization-aware training with straight-through rounding on $\rho$ and $a$, followed by a post-training pass of learnable rounding that minimizes ranking loss on held-out edges. Because $\rho$ and $a$ are always valid by construction, rounding can never push a point off the manifold, which removes the constraint problem raised in the file's Gap 4.

The third component is a distance-based decoder. This is the Fermi-Dirac link-prediction head used by Hyperbolic Graph Convolutional Neural Networks. We apply exponent coding only to the final node embeddings and the distance, and leave the message-passing layers in their usual precision. This isolates the claim to the part of the model where hyperbolic distance is the output.

The fourth component is a hardware realization. We will write register-transfer-level code for the distance unit, synthesize it for an FPGA, and report LUT, DSP-block and memory utilization. This matters because FLightNNs shows that on FPGAs, avoiding DSP-block multipliers is where the gains come from.

Baselines

The naive reference is the FP32 model: Poincaré embeddings trained as in Poincaré Embeddings, which report WordNet noun reconstruction mean average precision (MAP) of 0.823 at 5 dimensions and HyperLex Spearman correlation of 0.512. It is the accuracy ceiling. We will retrain it ourselves with the same data split, epochs and dimensions as our method, so the ceiling is measured rather than quoted. Beating it is not the goal. Matching it within 0.01 MAP at a stated bit budget is.

The strongest competitor on the actual question is a best-effort low-bit version of the standard coordinates. We will apply LogART-style learnable logarithmic rounding (LogART) and uniform learned-step integer quantization, each in two settings: to Poincaré coordinates, and to Lorentz coordinates with a power-of-two time coordinate. Each will get the same quantization-aware fine-tuning epochs and the same hyperparameter search budget as our method. No reported result was found on Lacuna for any quantized hyperbolic embedding on WordNet, so this baseline must be produced here. Our method beats it if, at equal bits per entity, it retains higher MAP. We expect the gap to widen as bit width drops and as we move to deeper subtrees, which we will measure by stratifying MAP by node depth.

A half-precision floating-point baseline (FP16 and bfloat16) in both models tests whether a hardware-native exponent-plus-mantissa format already solves the problem.

A Euclidean low-bit baseline trains 200-dimensional Euclidean embeddings and power-of-two quantizes them. Nickel and Kiela report FP32 MAP of 0.168 at that size. This sets what flat geometry buys under the same compression.

Two ablations isolate the key ingredients. The first uses the polar representation with FP32 distance. It separates the gain from choosing the right coordinates from any loss due to table-based arithmetic. The second replaces the logarithmic radius code with a uniform radius code. It tests whether "logarithmic in $\sinh r$" specifically matters. A third check removes the log-domain addition table and keeps only the maximum of the two terms. That tests whether the cheap tree-metric approximation $d \approx r_u + r_v + \log\sin^2(\theta/2)$ suffices.

Why not just…?

Why not just use INT8 on Lorentz coordinates with ordinary integer multipliers? This is the most serious threat, and it may work for shallow hierarchies. The cancellation argument predicts failure specifically for short distances between deep nodes. The mechanism is that relative error on terms of size $e^{2r}$ exceeds their order-one difference once $r$ passes roughly half the mantissa bits times $\ln 2$. The depth-stratified comparison in the baselines will settle this. If INT8 Lorentz matches us on WordNet at every depth, our contribution shrinks to the multiplierless hardware savings. That is a narrower claim, and we would say so. The hardware case rests on evidence that shift-based arithmetic elements use 43.2% less area and 61.2% less power than linear ones (LogART).

Why not just use FP16 or bfloat16? Half precision is itself an exponent plus a short mantissa. In the Poincaré model, however, its roughly $2^{-11}$ resolution on $1-\|x\|$ caps representable radius at a modest value. In the Lorentz model it inherits the same cancellation problem. It also still requires floating-point units, which the target hardware (Sorbet's neuromorphic setting, microcontrollers) does not have. The half-precision baseline measures the accuracy side directly.

Why not just use a flat embedding with standard power-of-two quantization? At full precision the geometry gap is already large: 0.823 versus 0.168 MAP at 5 versus 200 dimensions in Poincaré Embeddings. A 200-dimensional code at 4 bits costs 800 bits per entity. We will report MAP against bits per entity for both geometries on the same plot, so the comparison does not depend on this argument alone.

Why not just keep codebook indices, as in HyperVQ or residual quantization, and precompute the distances? Codebooks compress storage but still evaluate distances in FP32 (HyperVQ, Hyperbolic Residual Quantization). A precomputed distance table scales quadratically with codebook size. We will include a codebook-plus-table variant at matched memory as an extra point on the bits-versus-accuracy curve.

Proposed Experiments

The primary benchmark is WordNet noun reconstruction and link prediction at dimensions 2, 5 and 10, with MAP and mean rank as metrics. Results are stratified by node depth and swept over total bits per entity from about 16 to 64. HyperLex Spearman correlation tests whether graded entailment survives quantization, since it depends on norm ordering, which our radius code preserves by construction. DISEASE and AIRPORT link prediction with an HGCN encoder and an exponent-coded decoder, using ROC AUC, test transfer beyond pure embedding tables. There, the file reports only relative figures for HGCN (a 63.1% error reduction on DISEASE), so we will measure the FP32 HGCN reference ourselves.

The decisive checks are three. The first is whether our method retains near-FP32 MAP at 4 radius bits while quantized Poincaré and Lorentz baselines degrade with depth. The second is whether, in the FP32-polar ablation, the entire remaining gap comes from direction bits, confirming angular capacity as the bottleneck. The third is whether the synthesized distance unit uses zero DSP multipliers at FPGA resource and latency costs competitive with an INT8 multiply-based unit. If the first check fails, the representation argument is wrong. If only the third fails, the result is still a precision finding but not a hardware one.