Pharmaceutical companies hold reaction data they will not share, and the models trained on that data are just as valuable as the data. A natural way to serve such a model to partners without exposing either the model or the partner's target molecule is a trusted execution environment (TEE): a hardware-isolated region of the processor, called an enclave, whose memory the host operating system cannot read. The best retrosynthesis models no longer rely on reaction templates, the fixed rule books of known transformations. Retrosynthesis means predicting which starting materials make a given molecule, and the newer models instead generate the precursor molecules atom by atom and bond by bond. That step-by-step generation is a poor fit for enclaves. Enclaves are small, so heavy layers have to run outside on an untrusted GPU, and every hop across the enclave boundary is expensive. Each generation step therefore pays for several boundary crossings. In this setting, the latency of a single prediction is set mainly by how many times the computation crosses the boundary, not by how much arithmetic the model does.
We propose to treat the number of boundary crossings per molecule as a design variable. The work does two things. It trains a template-free graph decoder that emits several graph edits per enclave call instead of one. It then searches jointly over that block size and over which layers run inside versus outside the enclave, using a cost model calibrated on real hardware. The claim is that this joint search reaches a latency–accuracy trade-off that neither a hand-chosen layer split nor middleware batching can reach. It also keeps the main advantage of template-free models, which is predicting reactions whose template was never seen in training.
Core Claim
Consider any fixed partition of a model between enclave and host. Let $N(k)$ be the number of decoding calls needed to produce the precursors when the decoder emits $k$ edits per call, let $m$ be the number of enclave–host crossings per call under the chosen placement, let $c_\times$ be the measured cost of one crossing, and let $C_{\text{comp}}$ be the compute time. Per-query latency is then roughly
$$T \approx N(k)\, m\, c_\times + C_{\text{comp}}, \qquad \text{subject to } M_{\text{enclave}} \le M_{\text{budget}},$$
where $M_{\text{enclave}}$ is the memory held inside the enclave and $M_{\text{budget}}$ is the hardware limit (90–128MB of usable enclave memory on older Intel SGX, under 16–32MB on Arm TrustZone, per SoK: Machine Learning with Confidential Computing). The equation is simple accounting. Its point is that existing partitioning work optimizes $m$ and the memory term while treating $N$ as fixed by the model, and existing retrosynthesis work treats $N$ as a modeling detail. Autoregressive graph decoders such as NAG2G make $N$ grow with molecule size. Entering an enclave costs up to two orders of magnitude more than a function call, as measured in SecureStreams. The product $N m c_\times$ is therefore the term worth attacking. We claim that $N$ can be cut by several times through block decoding without giving up template-free accuracy, and that the best placement depends on $k$. Choosing the two jointly is what makes the method work.
How This Relates to Prior Work
Three TEE systems motivate the problem and set the bar.
DarkneTZ places the final layers of a vision classifier in TrustZone at 3–10% overhead. Its hand-picked cut point assumes a single forward pass, so it does not transfer to step-by-step generation.
MirrorNet automates part of the split. A strategy generator scales the channels of an enclave-side companion network to fit memory, guided by a performance emulator with 0.48% error. That search menu has no axis for how often the boundary is crossed.
Goten outsources linear algebra from SGX to an untrusted GPU using a triplet-based masking protocol, which hides tensors from the host. It reports 6.84x over TEE-only training, and it identifies cross-boundary communication as its remaining bottleneck. We reuse Goten's masking idea for the outsourced layers rather than inventing a new protocol.
The co-design and architecture-search literature supplies the search method and also constrains it. Auto-NBA relaxes network and accelerator choices into one differentiable search. DNArch sculpts CNN structure by backpropagation. Both work inside fixed human macro-templates, and both are blind to enclave boundaries. Our search is also a predefined menu, and we say so. The argument is that it adds an axis that was missing (decoding granularity tied to placement). It does not claim to escape predefined search spaces in general.
The chemistry side supplies both the reason to stay template-free and the reason not to. GLN shows that template models assign zero probability to any reaction outside their library. A template model is nonetheless naturally cheap in crossings, since one template application is one call. G2Gs is already partly one-shot: it predicts the reaction center in one step and then completes the resulting fragments. This is evidence that coarse-grained decoding can keep accuracy. CKIF is the closest privacy-aware retrosynthesis work. It is software federated learning on text-string (SMILES) models and addresses training across data silos, not confidential serving. Our work is complementary, and it targets graph models.
Research Program
The threat model has three parties. The model owner's weights and the client's target molecule must both stay confidential from an untrusted host that controls the operating system and the GPU. Embedding tables, the decoding controller, the nonlinear layers, and the beam state live in the enclave. Large matrix multiplications may be outsourced under Goten-style masking, so the host sees only masked tensors.
We will build three components.
The first is a block-decoding variant of a node-aligned graph-to-graph decoder in the style of NAG2G, with a lighter encoder than Uni-Mol+ so that it fits in enclave memory. At each call the decoder predicts up to $k$ atoms and bonds in parallel, conditioned on the partial graph. We train it with the usual atom-mapping-based node alignment, which NAG2G's ablation shows is essential (removing it cut Top-10 accuracy by nearly half). Training covers $k \in \{1, 2, 4, 8\}$ and a "whole graph in one shot" endpoint.
The second is a latency cost model. Its per-crossing cost $c_\times$, masking costs, and paging penalties are measured by microbenchmarks on the target hardware rather than taken from the literature. It will be validated against measured end-to-end latency to the accuracy standard MirrorNet's emulator sets.
The third is a search over three choices: the per-layer placement (inside the enclave, or outsourced under masking), the block size $k$, and whether the encoder output is cached inside the enclave across calls. The objective is predicted latency plus an accuracy penalty, subject to the memory budget. The search space is small enough to enumerate exhaustively after a pruning step, so no differentiable relaxation is needed. We prefer exact search here because it removes the gap between the relaxed surrogate and the discrete deployed choice that Auto-NBA-style methods must manage.
A secondary benefit comes for free. Padding every query to a fixed number of calls hides molecule size from anyone timing the enclave. Larger $k$ makes this padding cheap. We will measure that leakage reduction but not lead with it.
Baselines
The naive reference point is the reproduced block-size-1 decoder running entirely inside the enclave, with no outsourcing, paging whenever memory is exceeded. It shows what confidentiality costs before any engineering.
The strongest existing accuracy reference is NAG2G at 55.1% Top-1 and 99.7% validity on USPTO-50k with the reaction class unknown (NAG2G). Our lighter encoder will not match Uni-Mol+. To keep the comparison fair, we therefore treat our own reproduced block-size-1 model, with the same encoder, data, and training budget, as the accuracy anchor. Each block-decoded variant is judged against that anchor: losing no more than two Top-1 points counts as acceptable.
The strongest existing systems reference is a Goten-style partition applied to the block-size-1 decoder. Linear layers are outsourced under masking, everything else stays in the enclave, and beam candidates are batched within each call. This is the configuration a competent systems engineer would build today. It uses the same hardware, masking protocol, and beam width as our method. Goten's own reported gain is 6.84x over TEE-only training. We found no reported result on Lacuna for enclave latency of any retrosynthesis model, so this baseline must be measured by us. Our method beats it if it cuts median single-query latency by at least 3x at matched accuracy. Single-query latency matters because a chemist querying interactively cannot wait to be batched with other requests.
The template reference is GLN, with its template library kept in the enclave. Its reported Top-1 is 52.5% on USPTO-50k with the reaction class unknown (GLN). It represents the "few crossings, bounded generalization" corner of the trade-off.
Two ablations isolate the key ingredient. Placement-only search at $k=1$ is a MirrorNet-like menu extended to graph decoders. Block-size-only search fixes placement at the Goten split. If either one alone reaches the joint optimum, the claim that the two must be chosen together fails.
Why not just…?
Why not just run the whole model inside the enclave? On hardware where the model fits, such as larger-memory SGX or an Arm CCA Realm, crossings drop to one per query, and the remaining cost is the monitor overhead that An Early Experience With Confidential Computing Architecture for On-Device Model Protection measures at 17–22% of instructions. This is a real threat. It loses only if CPU-only compute inside the enclave is slower than GPU outsourcing plus crossings, which Goten's 6.84x suggests for training but nobody has shown for small-batch graph decoding. We run this configuration on every platform. If it wins wherever the model fits, we report that partitioning helps only under tight memory budgets such as TrustZone, and the contribution narrows to that regime.
Why not just batch more aggressively? SecureStreams shows that chunking amortizes crossings, with gains flattening around 64KB per chunk. All our variants already batch the beam within each call, so that gain is shared. Batching across separate queries cuts throughput cost but adds queueing delay, so we report both single-query latency and saturated throughput. If batching across queries closes the gap on throughput, we will say that block decoding is a latency method only.
Why not just use a template model or a one-shot predictor? Both make $N$ tiny. Templates pay with zero probability for any reaction outside the library. The decisive test is accuracy on test reactions whose extracted template never appears in training, where GLN scores zero by construction. If a fully one-shot, G2Gs-like decoder ($k$ equal to the whole graph) matches block decoding in accuracy, the search collapses to a trivial answer. That would itself be a useful finding, and the $k$ sweep will show it directly.
Why not use homomorphic encryption or secure multiparty computation instead of hardware? Goten reports 132.64x over the Falcon multiparty-computation system, and PRESAGE reports 120x for enclaves over homomorphic encryption. Step-by-step decoding would multiply those costs further, so we do not pursue this.
Proposed Experiments
Accuracy is measured on USPTO-50k with the reaction class both known and unknown (Top-1/3/5/10 and validity), and on USPTO-full for scale. We also build an out-of-template split of USPTO-50k, holding out every test reaction whose extracted template is absent from the training reactions. Systems measurements run on an SGX server paired with an untrusted GPU, and on the Arm CCA Fixed Virtual Platform for instruction counts comparable to Abdollahi et al. We report median and 95th-percentile single-query latency, crossings per query, enclave memory, and throughput.
Three checks decide the proposal. First, the measured latency-versus-accuracy curve of the joint search must dominate both ablations and the Goten-style baseline: at least 3x lower single-query latency within two Top-1 points of the block-size-1 anchor. Second, on the out-of-template split, the chosen block-decoded model must keep most of the anchor's accuracy while GLN scores zero. This shows the speedup was not bought by quietly collapsing back to template-like behavior. Third, the cost model must predict measured latency closely enough to rank configurations correctly. If its ranking disagrees with measurement, the search result is not trustworthy, and we fall back to exhaustive measurement on hardware.
References
- Machine Learning with Confidential Computing: A Systematization of Knowledge
- Node-Aligned Graph-to-Graph: Elevating Template-free Deep Learning Approaches in Single-Step Retrosynthesis
- SecureStreams: A Reactive Middleware Framework for Secure Data Stream Processing
- DarkneTZ: towards model privacy at the edge using trusted execution environments
- MirrorNet: A TEE-Friendly Framework for Secure On-Device DNN Inference
- Goten: GPU-Outsourcing Trusted Execution of Neural Network Training
- Auto-NBA: Efficient and Effective Search Over the Joint Space of Networks, Bitwidths, and Accelerators
- DNArch: Learning Convolutional Neural Architectures by Backpropagation
- Retrosynthesis Prediction with Conditional Graph Logic Network
- A Graph to Graphs Framework for Retrosynthesis Prediction
- Chemical knowledge-informed framework for privacy-aware retrosynthesis learning
- An Early Experience With Confidential Computing Architecture for On-Device Model Protection
- PRESAGE: PRivacy-preserving gEnetic testing via SoftwAre Guard Extension