Two programs can do exactly the same thing and still look very different. One sorts recursively and the other with a loop. One uses for and the other while. One carries dead code the other lacks. Hyperbolic embeddings, which place data in a curved space whose volume grows exponentially outward like a tree, are good at representing the nested structure of code's syntax trees. Because they are faithful to that structure, though, they also place structurally different implementations of the same function far apart. The authors of HypeCodeNet report exactly this failure on recursive-versus-iterative pairs. The usual fix is contrastive training on semantics-preserving rewrites, as in ContraCode: pull a program and its rewrite together. Applied naively in hyperbolic space, that fix should defeat the purpose. If a program and its structurally deeper rewrite are forced to the same point, the encoder has to stop encoding depth, and the hierarchy that justified the curved geometry disappears.
We propose splitting the job geometrically. In the Poincaré ball, a hyperbolic space drawn as the inside of a unit ball, each embedding has a distance from the origin (its radius) and a direction. Prior hyperbolic work reads radius as specificity or depth, with general concepts near the center and specific leaves near the edge; Learning the Predictability of the Future uses the same reading when it pulls uncertain predictions toward the origin. We train a Siamese (twin, weight-shared) hyperbolic encoder of syntax trees on correct-by-construction program rewrites. The contrastive loss constrains only direction, and radius stays free to track structural depth. The claim is testable: this encoder should match or beat Euclidean and full-hyperbolic-distance contrastive models at recognizing structurally divergent clones, while keeping the low tree distortion that full-distance contrastive training destroys.
Core Claim
Write an encoder output as a point $z$ in the Poincaré ball of curvature $-1$. Its radius is $\rho(z) = 2\,\mathrm{artanh}(\lVert z\rVert)$ and its direction is $u(z) = z/\lVert z\rVert$. Because the ball is conformal, the direction of $z$ equals the direction of its logarithmic map at the origin, so direction can be compared by plain cosine similarity. For a program $x$ and a rewrite $x^+ = T(x)$ drawn from a family of semantics-preserving transforms $T$, with negatives $x^-_j$ from a queue, the loss is
$$\mathcal{L}_{\text{dir}} = -\log \frac{\exp(u(z)^\top u(z^+)/\tau)}{\exp(u(z)^\top u(z^+)/\tau) + \sum_j \exp(u(z)^\top u(z^-_j)/\tau)},$$
where $\tau$ is a temperature. Radius gets no contrastive pressure. It is shaped only by the tree-structural objective the encoder already uses, which is HypeCodeNet-style message passing that reconstructs structure. The contrasting design is the same InfoNCE loss computed with full geodesic distance $d_{\mathbb{D}}(z, z^+)$.
We make two predictions. First, full-geodesic contrastive training shrinks the spread of radii across programs and raises tree distortion toward Euclidean levels. It has to, because rewrites like recursion-to-iteration or dead-code insertion change tree depth, and pulling their endpoints together penalizes any radius that tracks depth. Second, direction-only training avoids this collapse while giving at least equal clone recognition on structurally divergent pairs. If the first prediction fails, the proposal's motivation is wrong. If the second fails against a Euclidean baseline, curvature is not buying invariance.
Where This Sits
The four input directions play different roles here. Semantics-preserving augmentation supplies the training signal. We reuse lexical rewrites (renaming, dead code, statement reordering) studied by Jain et al. and Liu et al., structural rewrites from NatGen (loop interchange, branch swaps), and recursion-to-iteration conversion from Chen et al.. Compiler-pass variants in the style of TehraniJamsaz et al. and Ma et al. are an optional source of large structural change.
NatGen also constrains the design. It found a formulaic dead-code pattern that models solved 99% of the time. It also found that models facing stacked transforms fix only the most salient one. We therefore hold out entire transform families at test time rather than trusting in-distribution rewrites.
The Siamese direction supplies the architecture. Shared weights, as in Bromley et al. and the clone detector Oreo, make similarity symmetric by construction. Oreo compares 24 hand-made software metrics in flat space. We compare learned tree embeddings in curved space.
Hierarchical embedding distortion is the property we are trying to protect. HypeCodeNet reports more than 60% lower distortion on deep syntax trees and matches 768 Euclidean dimensions with 32 hyperbolic ones, but it uses no rewrites and no pairwise metric training. HELoC forces Euclidean distance to track tree-level differences. Hyperbolic Function Embedding shows that call graphs have negative curvature. Neither combines hierarchy with rewrite invariance.
Advanced augmentation, specifically HypMix, which performs Mixup with Möbius operations inside the Poincaré ball, enters as the strongest competing augmentation rather than as part of the method. Mixing two unrelated programs gives no semantic-equivalence signal, so it tests whether the gains come from the rewrites or merely from geometric regularization.
One paper runs against the premise. Fu et al. report that flat document embeddings reach 96.3% Type-4 F1 on BigCloneBench and argue that tree structure hurts semantic clone detection. Our diagnosis is compatible with theirs: tree structure hurts when the metric forces syntactic depth to count as semantic difference. The direction/radius split is our proposed remedy, and a flat document-embedding baseline stays in the comparison.
Research Program
The encoder is a Lorentz- or Poincaré-model graph network over syntax trees. It uses the log-aggregate-exp message passing and curvature annealing of HypeCodeNet and is optimized with Riemannian Adam. Its root-pooled output is the program embedding. Pretraining uses a subset of Java-Large with a momentum queue of negatives, as in ContraCode.
For each program we sample one or two transforms, verify equivalence by compiling the rewrite and checking agreement on tests where they exist, and build the positive pair.
Beyond task accuracy we measure three things. The first is AST node-embedding distortion, the average relative error between embedding distances and tree distances, following HypeCodeNet. The second is the spread of program radii. The third is the correlation between radius and syntax-tree depth.
The central analysis follows these quantities through training under the direction-only and full-geodesic losses, across a curvature sweep that reaches the Euclidean limit. The result is one curve per loss showing invariance gained against hierarchy lost.
Baselines
The naive reference points are a hyperbolic tree encoder trained only on its structural objective, with no rewrites (essentially HypeCodeNet), and CodeBERT fine-tuned on pairs. On BigCloneBench, CodeXGLUE reports CodeBERT at F1 96.5 and HypeCodeNet reports F1 0.940. These show how much rewrite training adds over no invariance training at all.
The strongest reported number on BigCloneBench in the file is HELoC at F1 98.0%. Since this benchmark is nearly saturated, beating it there is not our main criterion. We quote it and aim only to stay within about a point, while the decisive comparisons happen on harder splits.
The strongest competitor on the actual question is a Euclidean Siamese model with the same tree encoder, curvature fixed at zero, trained with ContraCode's InfoNCE on the identical rewrite stream. It shares data, queue size, number of steps, and a hyperparameter sweep of equal size (we plan twenty configurations per model), so the only difference is geometry. We give it both its native embedding width and a matched parameter count. For ContraCode's JavaScript adversarial retrieval, the file reports only a relative gain of +39% AUROC over RoBERTa, so we rerun ContraCode rather than quote an absolute number.
The key ablation is the full-geodesic hyperbolic contrastive model, identical to ours except that the loss uses $d_{\mathbb{D}}$ instead of direction. A second ablation replaces the rewrites with HypMix interpolation in the same Siamese network. For POJ-104 and OJClone, no reported result was found on Lacuna, so all numbers there come from our own reruns.
We count the proposal as winning if two conditions hold together over five seeds. On held-out-transform and divergent-pair splits, the direction-only model must beat the Euclidean contrastive model by more than twice the seed standard deviation in MAP@R or AUROC. Its tree distortion must also stay within 10% of the unaugmented hyperbolic encoder, while the full-geodesic model's distortion rises measurably.
Why not just…?
Why not just train ContraCode-style contrastive learning with a Euclidean tree encoder? This is the real threat. Euclidean contrastive training may deliver all the invariance, with curvature adding only compression. Nothing in the file shows otherwise, since no paper compares geometries under rewrite training. The matched Euclidean baseline and the curvature sweep settle it. If the Euclidean model ties on divergent pairs, we report that curvature's value lies in dimension efficiency and hierarchy-dependent tasks, not in invariance.
Why not just use full hyperbolic distance in the contrastive loss? It is the natural default. Our argument is that it collapses radius, because depth-changing rewrites are positives. This is the paper's first prediction, and the ablation measures it directly through radius spread and distortion. If full-geodesic training keeps distortion low and matches retrieval, the split is unnecessary and we say so.
Why not just apply HypMix, or even CutMix-style mixing, instead of building rewrite tools? Cut to the Mix found that crude patch mixing beat elaborate, domain-aware augmentation in low-data segmentation. HypMix cut error from 18.8% to 16.5% on 5-shot Urdu speech emotion recognition versus Euclidean Mixup. Cheap geometric regularization therefore deserves a fair test. Mixing unrelated programs, however, never tells the model that for and while versions are the same function, which is the specific failure HypeCodeNet reports. The HypMix ablation on the held-out-transform split shows whether regularization alone closes the gap.
Why not just fine-tune a large code language model on augmented clone pairs? A fine-tuned CodeBERT is included as a baseline. What it cannot give is an interpretable separation between how deep the code is and what the code does, which is where our distortion and radius analyses apply. If it wins on retrieval, the geometric claim must rest on distortion and efficiency alone.
Proposed Experiments
Training uses Java rewrites with NatGen- and Chen-style tools. Evaluation runs on BigCloneBench (F1, reported by clone type), POJ-104 and OJClone (MAP@R), and two constructed splits.
The first constructed split is a divergent-pair split. It takes the BigCloneBench and POJ-104 clone pairs in the top quartile of tree-edit distance or depth difference, the regime where HypeCodeNet struggles.
The second is a held-out-family split. Recursion-to-iteration and compiler-pass variants are never seen in training and are applied at test time with one to sixteen stacked edits, mirroring ContraCode's adversarial protocol.
A JavaScript replication on ContraCode's adversarial retrieval checks that the result carries across languages. Hierarchy retention is measured by node-level distortion, a probe that predicts syntax-tree depth from radius, and GitHub Java Corpus link prediction, where HypeCodeNet reports AUC 0.965.
The decisive check is a joint plot of divergent-pair AUROC against tree distortion for the direction-only, full-geodesic, and Euclidean models across curvature values. The proposal holds only if the direction-only model sits alone in the high-invariance, low-distortion corner. A secondary check verifies the proposed semantics of radius: on held-out rewrites, the difference in radius between a program and its rewrite should correlate with the difference in their tree depths, while their directions stay aligned.
References
- The Natural Geometry of Code: Hyperbolic Representation Learning for Program Reasoning
- Contrastive Code Representation Learning
- Learning the Predictability of the Future
- ContraBERT: Enhancing Code Pre-trained Models via Contrastive Learning
- NatGen: generative pre-training by "naturalizing" source code
- Chain of Execution Supervision Promotes General Reasoning in Large Language Models
- Learning Intermediate Representations using Graph Neural Networks for NUMA and Prefetchers Optimization
- Unleashing the Power of Compiler Intermediate Representation to Enhance Neural Program Embeddings
- Signature Verification Using a Siamese Time Delay Neural Network
- Oreo: detection of clones in the twilight zone
- HELoC: hierarchical contrastive learning of source code representation
- Hyperbolic Function Embedding: Learning Hierarchical Representation for Functions of Source Code in Hyperbolic Space
- HypMix: Hyperbolic Interpolative Data Augmentation
- An ensemble learning approach for software semantic clone detection
- CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation
- Cut to the Mix: Simple Data Augmentation Outperforms Elaborate Ones in Limited Organ Segmentation Datasets