Code-completion tools that suggest the next method call work well for popular library APIs and fail badly on the functions a project defines for itself. The APIBench study found that success rates of eleven recommenders drop by 35.3% to 91.3% when the target is a user-defined API (Revisiting, Benchmarking and Exploring API Recommendation: How Far Are We?). This is a cold-start problem in the recommender-systems sense. A function written ten minutes ago has no usage history, so nothing learned from co-usage statistics applies to it. Yet in a real codebase these private helpers are a large share of the calls a developer actually makes. One thing is always available for them that is never available for a new movie or product: the function's own source code.
The proposal is to learn, from library APIs that already have plenty of call sites, a mapping from an API's implementation to the kinds of calling contexts it appears in. That mapping is then applied zero-shot to user-defined functions. Concretely, a two-tower model scores each pair of calling context and candidate API. One tower reads the partial code at the cursor. The other reads only the candidate's definition: its signature and the structure of its body, meaning what it reads, what it returns, and which other APIs it calls. Because the API tower never sees a learned per-API identity vector, a function with zero calls gets a meaningful score. As a few calls accumulate inside the project, a small per-API usage correction is blended in. The new claim is about where the information comes from. For cold APIs we expect the body's data flow and internal calls, not the name, to carry most of the signal about usage. A renaming test, described below, can falsify that claim directly.
What Exists and What Is Missing
Two lines of work handle cold-start APIs through text alone. APIRecX splits method names into subword pieces with a GPT-style decoder, which lets it predict calls never seen in training. It reaches 52.9% top-1 on the IO domain with 0.2% fine-tuning data, and 28.1% of its correct predictions are out-of-vocabulary APIs. Its knowledge of an unseen API is therefore entirely lexical: if the name is uninformative, it has nothing to go on. Req2Lib maps requirement text to library identifiers before any code exists. That covers project inception, not functions defined mid-development. BRAID re-ranks query-based suggestions with implicit feedback and bootstraps from zero interactions through active learning against Stack Overflow. It works on documentation text and has nothing to say about a private helper with no documentation and no Stack Overflow presence.
Structural code representations supply our method. code2vec shows that attention over paths in a program's syntax tree yields a fixed vector predictive of a method's purpose. It is trained to predict method names, which is exactly the lexical shortcut we want to test against. Neural Code Comprehension shows that graphs combining data flow and control flow capture behaviour that flat sequences miss. Its compiler-level representation of Java and Python is not practical in an editor, so we use source-level data-flow graphs instead.
Recommender-systems work constrains the design. DisCo finds that forcing content embeddings to match collaborative embeddings distorts the content signal, and it gains on tail items (AUC 0.6673 to 0.7416 on AZ-Toys) by keeping the two spaces separate. That result is why our usage correction is an additive residual and not an alignment target for the content tower. SimGCL shows that pushing embeddings toward uniformity on the sphere with simple embedding noise reduces popularity collapse. We reuse it to stop the API tower from mapping every helper onto toString-like popular calls. Xu et al. provide a warning: text-only content models failed to beat plain matrix factorization when interaction data exists. We take that seriously, and it is why the main evaluation is stratified by how many calls an API already has.
Core Claim
For an API with zero or a few observed uses in a project, an embedding computed from its implementation's structure predicts its call sites better than an embedding computed from its name or documentation. The advantage should widen when identifiers are uninformative. The scoring function is
$$s(c, a) = f(c)^\top \big( g(\mathrm{def}_a) + \lambda_{n_a}\, u_a \big),$$
where $c$ is the calling context, $f$ the context tower, $\mathrm{def}_a$ the source definition of candidate API $a$, $g$ the definition tower, $u_a$ a per-API usage vector learned from in-project calls, $n_a$ the number of observed calls to $a$, and $\lambda_{n_a}$ a gate that is zero when $n_a=0$ and grows with $n_a$. Training uses an in-batch contrastive loss over warm library APIs. With probability $p$, $u_a$ is dropped during training, so that $g$ alone must carry the signal. SimGCL-style noise on $g(\mathrm{def}_a)$ supplies the uniformity term.
Research Program
The data come from APIBench-C (1,477 Java and 2,223 Python repositories). We build a cold-start split inside each held-out project. Every call site of a user-defined method is a test query, and the method's definition is available to the model. For training, we take third-party and standard-library APIs whose source we can resolve and that have many call sites across training repositories. This teaches $g$ what implementations of used things look like. The definition tower encodes a data-flow graph over the body: parameter-to-return paths, field reads and writes, and the identities of internally called library APIs. The internally called APIs are themselves warm, so a new helper inherits usage signal from the calls it wraps. The context tower is a standard pretrained code encoder over the preceding code.
We stratify evaluation by $n_a \in \{0, 1\text{–}2, 3\text{–}5, >5\}$ and by position in the function, since APIBench reports a 10.6% drop at the start of functions. The central diagnostic is identifier anonymization. At test time we rename user-defined methods and their parameters to opaque tokens while leaving the body's structure intact. A name-based model should collapse under this renaming, and a structure-based one should not.
Baselines
The naive reference is in-project frequency: rank user-defined and library APIs by how often they have been called so far in the file and project, falling back to global library popularity. For $n_a=0$ this cannot rank the target at all, and it sets the floor every method must clear.
The strongest published code-based competitor is TravTrans, the code-based model APIBench evaluates. APIBench reports 0.62 Success Rate@10 for it on Python codebases overall. We found no reported result on Lacuna for TravTrans, or any other system, as an absolute number on the user-defined split; only the 35.3%–91.3% relative drop is given. We will rerun it on our split with its released configuration.
The strongest cold-start competitor is an APIRecX-style subword decoder pretrained on the same training repositories and fine-tuned with the same budget. It is the established answer to unseen APIs. A further dense-retrieval baseline encodes each user-defined method's signature and docstring with a CodeBERT bi-encoder (CodeXGLUE) and ranks by similarity to the context. This is the plainest form of "content features for cold start."
All baselines see the same repositories and the same definitions where they can use them, and get the same hyperparameter-search budget measured in GPU-hours. Two ablations isolate the key ingredient. The first restricts the definition tower to the signature only, with no body. The second removes the internal-call features. Beating the field means a statistically significant gain in Success Rate@10 at $n_a=0$ over the best of TravTrans, the subword decoder and the bi-encoder, with the gap at least maintained under identifier anonymization. Paired bootstrap over projects will establish significance.
Why not just…?
Why not just embed the signature and docstring with a pretrained code encoder? For well-named helpers this may come close, and our signature-only ablation is exactly this test. User-defined functions often lack docstrings, and names like process or handle2 say little. The anonymization condition shows how much of any text-based success rests on naming. If the signature-only model matches the full model even after anonymization, the structural claim is wrong and we will report that.
Why not just use a subword language model, which already handles unseen names? APIRecX shows subwords recover many unseen APIs, but every piece of evidence it uses about a new API is in the name. Neural Code Comprehension shows that flat sequences miss behavioural equivalence across renamings and reorderings. The renaming test is designed to separate the two approaches, and we expect the subword model to degrade most there.
Why not just paste the project's definitions into a large code model's prompt? This is the most serious threat. Retrieval-augmented completion with a large model may simply win on accuracy. We will run it as a reference, retrieving the top definitions into the prompt, and report accuracy alongside latency, because in-editor ranking must work at the millisecond scale that Xu et al. show separates interaction models from heavy text models. We will also measure how often it calls functions that do not exist, the action bias documented by Yang et al.. If it beats us at comparable cost, our contribution narrows to a cheap structural reranker that supplies the candidate list such a model retrieves from. We would test that combination rather than claim superiority.
Why not just standard collaborative filtering with a popularity penalty? With $n_a=0$ there is no row to factorize. APIBench's collapse figures already show what happens to models that depend on usage history. That approach becomes relevant only in the $n_a>5$ stratum, where we expect it, and our gated $u_a$ term, to catch up.
Proposed Experiments and Decisive Checks
The main experiment reports Success Rate@1 and @10 and MRR on the APIBench-C Java and Python cold-start splits, stratified by $n_a$ and by position in the function. The results should be read as curves showing where structural content stops mattering and usage takes over.
Three checks decide the claim. The first is anonymization: the full model must keep most of its $n_a=0$ accuracy when names are scrambled, while the subword and bi-encoder baselines should not. The second checks the DisCo-motivated design choice. Replacing the additive residual with direct alignment of $g$ to $u_a$ should hurt the cold strata. If it does not, the separation is unnecessary and we drop it. The third tests the uniformity term on popularity-bias histograms. Turning off SimGCL noise should increase how often generic library calls displace the correct user-defined target, measured as the share of top-10 slots taken by the 1% most popular APIs.
A negative result on the first check would be informative in its own right. It would mean that cold-start API recommendation is, in practice, a naming problem.
References
- Revisiting, Benchmarking and Exploring API Recommendation: How Far Are We?
- APIRecX: Cross-Library API Recommendation via Pre-Trained Language Model
- Req2Lib: A Semantic Neural Model for Software Library Recommendation
- Boosting API Recommendation With Implicit Feedback
- code2vec: learning distributed representations of code
- Neural Code Comprehension: A Learnable Representation of Code Semantics
- DisCo: Towards Harmonious Disentanglement and Collaboration between Tabular and Semantic Space for Recommendation
- Are Graph Augmentations Necessary?: Simple Graph Contrastive Learning for Recommendation
- Understanding the Effectiveness of Reviews in E-commerce Top-N Recommendation
- CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation
- Can Tool-augmented Large Language Models be Aware of Incomplete Conditions?