Constant-Depth Encrypted Trajectory Models via Time-Moment Accumulators

In chronic disease, the most useful signal is often how fast a measurement is changing, not its current value. The annual slope of kidney filtration, the 12-month change in bone-marrow fat, and trend lines in brain imaging all predict outcomes that single snapshots miss. These trajectories are also some of the most sensitive data a patient has, and they are usually spread across years and clinics. Homomorphic encryption, a family of cryptosystems that lets a server compute on data it cannot read, should fit this setting well. In practice it breaks on sequences. Each step of a recurrent model multiplies ciphertexts together, and every such multiplication consumes part of a fixed "noise budget." Unrolling over many visits therefore exhausts the budget and forces bootstrapping, an expensive refresh procedure. It can also push the model into approximations crude enough to break its predictions.

We propose to avoid sequential unrolling altogether. The idea is to restrict the temporal part of the model to a learned weighting of past visits, where the weighting is a low-degree polynomial in elapsed time. If you expand that polynomial, the model's summary of the whole history becomes a fixed linear combination of a few running sums of the form $\sum_i t_i^j x_i$. A clinic can encrypt and upload one visit's terms without ever seeing the rest of the patient's history, and the server adds them to an encrypted accumulator. At query time, the server combines the accumulators using plaintext coefficients that depend only on the current time. The number of multiplicative levels spent on history is one, and it stays one whether the patient has 3 visits or 300. Storage also does not grow with the number of visits. The research question is whether this constant-depth model class keeps most of the predictive value that richer sequence models such as GRU-D extract from irregular clinical time series.

Core Claim

Our hypothesis concerns the clinical tasks where trajectories matter. On those tasks, a readout that combines polynomial-kernel moment features with a shallow polynomial network should come close to a gated recurrent model with missingness handling. Under leveled CKKS encryption, it should need no bootstrapping, no ciphertext rotations for the temporal part, and a fixed multiplicative depth of about three to four. CKKS is the approximate-arithmetic homomorphic scheme used in Logistic Regression on Homomorphic Encrypted Data at Scale.

Several studies motivate this model class by showing that simple rates of change carry the signal. In Predicting Future Clinical Changes of MCI Patients Using Longitudinal and Multimodal Biomarkers, trend lines fitted to ADNI brain regions raised correlation with 24-month cognitive change from 0.63–0.66 (baseline only) to 0.786 (MMSE) and 0.777 (ADAS-Cog). In Associations of incidental vertebral fractures and longitudinal changes of MR–based proton density fat fraction, the 12-month change in marrow fat separated patients who went on to fracture, while baseline bone density did not. In Anemia is a risk factor for rapid eGFR decline in type 2 diabetes, annual eGFR slope defined the rapid progressors. Slopes, recency-weighted means, and observation counts are all linear functionals of the history, and every linear functional of a history with polynomial time weights can be written in moment form.

The Setting and Why It Needs Encryption of the Trajectory Itself

The scheme is only worth building when no single party holds the patient's plaintext history. We target an encrypted longitudinal record. Visits are produced at different clinics over years and encrypted at upload under the patient's key. They are stored by a cloud service that also owns a proprietary prognostic model. The clinic at visit $i$ knows only $x_i$ (the feature vector, with a missingness mask $m_i$) and the visit time $t_i$. It does not know the earlier visits. It cannot compute a slope locally, and it does not know the server's model weights.

The existing work at this intersection leaves exactly this case open. Encrypted Accelerated Least Squares Regression fits static ridge regression under encryption, including on a longitudinal mood dataset. We take its method lesson directly: its van Wijngaarden trick spends one level for a whole series instead of one per iteration, and we seek the same "one level for the whole history" property. It never models time. Privacy-Preserving Gaussian Process Regression serves as both motivation and constraint. It shows that polynomial approximation of nonlinear kernels under encryption can diverge catastrophically (±587,863% error), and it fixes this only through client round trips that reveal intermediate distances. We avoid approximating any nonlinear function at inference: the model is trained in its polynomial form from the start. A Homomorphic Encryption Framework for Privacy-Preserving Spiking Neural Networks is the only encrypted temporal model in the file. Its cost grows linearly with time steps (31 s to 930 s per image over 30 steps), which is the scaling we remove.

Research Program

The model. Rescale time to $s \in [-1, 1]$ over a fixed horizon, such as 5 years for ADNI or 48 hours for ICU data. The model's history summary for feature $f$, evaluated at query time $T$, is

$$h_{f,k}(T) = \sum_i \phi_k(T - s_i)\, m_{i,f}\, x_{i,f}, \qquad \phi_k(u) = \sum_{j=0}^{d} a_{k,j}\, P_j(u).$$

Here $\phi_k$ is the $k$-th learned temporal kernel, $P_j$ are Legendre polynomials, $a_{k,j}$ are trained weights, and $d$ is the kernel degree. Expanding $P_j(T - s_i)$ in powers of $s_i$ gives

$$h_{f,k}(T) = \sum_{j=0}^{d} c_{k,j}(T)\, M_{f,j}, \qquad M_{f,j} = \sum_i s_i^j\, m_{i,f}\, x_{i,f}.$$

The coefficients $c_{k,j}(T)$ are plaintext and depend only on the weights and $T$. The moments $M_{f,j}$ are the encrypted accumulators. A parallel set of mask moments $\sum_i s_i^j m_{i,f}$ gives the model access to how often and when a feature was measured. This preserves the informative-missingness signal that GRU-D exploits in Recurrent Neural Networks for Multivariate Time Series with Missing Values, but through counts rather than learned decay. The $h$ features feed a shallow readout that is trained end to end in plaintext: one or two layers with square activations, as in CryptoNets.

Upload and storage. The uploading clinic computes $s_i^j m_{i,f} x_{i,f}$ for $j \le d$ in plaintext and encrypts these terms. Visit times therefore never appear in the clear at the server, and the server never evaluates a polynomial on encrypted time. The server adds the new terms to the stored moments, which is a free operation. Patients occupy SIMD slots, meaning the thousands of parallel values packed into one ciphertext. Each (feature, moment) pair is a separate ciphertext. Temporal aggregation is then a slot-wise product with a plaintext vector, which resolves the packing-versus-sequence tension raised in the file without any rotations.

Division-free slopes. When the clinical question is a threshold on the slope, such as "rapid eGFR decline if the slope is below $-k$," the ordinary least-squares slope is a ratio of degree-2 expressions in the moments. Because the denominator is positive, the ratio test is equivalent to testing the sign of its numerator plus $k$ times its denominator. The server returns that encrypted score, and the key holder decrypts and thresholds it. No encrypted division is needed. This costs two levels instead of one, which is still constant in the number of visits.

Analysis. We will characterize three things. The first is what degree $d$ is needed to mimic exponential recency decay over each horizon. The second is the numerical conditioning of the moments under CKKS fixed-point precision as visit counts grow; Legendre scaling is meant to control it. The third is a structural limitation: a polynomial kernel cannot sharply ignore visits older than the horizon. Our remedy is to rotate accumulators into epoch blocks, which gives a piecewise kernel, and we will measure what that costs. Finally, a limitation we state rather than solve: hiding $t_i$ inside ciphertexts does not hide upload timing from the server. We will report the accuracy cost of padding with dummy uploads as a mitigation.

Baselines

The naive reference is an encrypted shallow model on the most recent visit alone, meaning the same square-activation readout on $x_T$ with no history. It is the cheapest thing anyone would deploy. The TADPOLE follow-up (Results after 1 Year Follow-up) reports that cross-sectional data cost the best methods only 2 percentage points on diagnosis prediction, so on some endpoints this baseline may be hard to beat. It gets the same readout architecture and the same hyperparameter search budget as our model. Beating it means a statistically significant gain on the slope-driven endpoints: MCI-to-AD conversion, 24-month cognitive change, and ICU mortality.

The accuracy ceiling is plaintext GRU-D, trained with its original masking and decay inputs on the same splits. On MIMIC-III its reported result is AUC 0.8527, versus 0.8252 for GRU-Mean (Recurrent Neural Networks for Multivariate Time Series with Missing Values). For PhysioNet 2012 we found no reported result on Lacuna, so we will rerun GRU-D ourselves. We count the proposal a success on MIMIC-III if the encrypted moment model lands between GRU-Mean and GRU-D, and within about one AUC point of our own GRU-D rerun. On ADNI/TADPOLE the plaintext reference is the trend-feature model with correlations 0.786/0.777 and 78.4% conversion accuracy (Predicting Future Clinical Changes of MCI Patients). That model used MRI and FDG-PET, so we will rerun a matched-feature version rather than compare against the published figure directly.

The strongest encrypted competitor is an encrypted recurrent model: a GRU with square or low-degree polynomial gates, retrained in that form and unrolled over visits under CKKS, with bootstrapping wherever depth demands it. Both it and our model get the same CKKS library, the same 128-bit security parameters, and the same hardware. We will report accuracy, depth, latency per batch, and key and ciphertext sizes. Beating it means comparable AUC at a latency and memory cost that does not grow with the number of visits.

Two ablations isolate the key ingredient. The first replaces the learned kernels $\phi_k$ with fixed hand-crafted functionals (OLS slope, mean, count), which tests whether learning the temporal weighting matters. The second removes the mask moments, which tests whether the missingness signal survives the moment form. We will also sweep the kernel degree $d$.

Why not just…?

Why not just use the latest visit and skip trajectories? Sometimes this will be enough, and we will say so. TADPOLE's 2-point cross-sectional gap for diagnosis suggests it may be sufficient for that endpoint. The bone and kidney results, and the ADNI correlation jump from about 0.65 to about 0.78, suggest it will not be enough for rate-driven endpoints. The last-visit baseline is run on every task, and our claim is scoped to the tasks where it loses.

Why not just have the client compute slopes and send them encrypted? If one trusted party holds the full plaintext history, this is simpler, and our method adds only the proprietary learned weighting. In the encrypted-record setting no uploader has the history, so local slopes are impossible. Even so, the fixed-functional ablation is exactly this alternative's feature set. If it matches the learned kernels, the honest conclusion is that the contribution reduces to the depth-1 accumulator protocol, not a better model.

Why not just run an encrypted GRU or GRU-D with bootstrapping? Its cost grows with the number of visits. The only temporal encrypted model in the file slowed 30-fold over 30 steps, and bootstrapped logistic regression alone took about 17 hours to train (Logistic Regression on Homomorphic Encrypted Data at Scale). GRU-D's exponential decay also needs polynomial approximation of the kind that failed badly in the encrypted GP work. This is the real accuracy threat, and the head-to-head comparison settles it.

Why not just use interactive computation or federated learning? The client-assisted GP protocol and SAFENet's hybrid evaluation (SAFENet) are accurate but need the key holder online for each query and expose intermediate values. Federated learning (Fluent) protects training, not inference on one patient's scattered history. Neither fits a record assembled by clinics that never see each other's data.

Proposed Experiments

We will run MIMIC-III and PhysioNet 2012 mortality prediction with GRU-D's preprocessing and splits. On TADPOLE D1/D2 we will predict MCI conversion and ventricle-volume change, and report ADAS-Cog13 only as a sanity check, since no method beat chance on it. We will define rapid eGFR decline on a nephrology-style cohort if access allows; otherwise we will use a TADPOLE-derived slope-threshold task for the division-free score.

The first decisive check is accuracy parity. The moment model, evaluated under actual CKKS encryption, must stay within about one AUC point of its own plaintext evaluation, which shows the cryptographic noise is harmless. It must also stay close to GRU-D. The second is scaling: latency and depth must be flat as we subsample histories from 5 to 200 observations, while the encrypted GRU grows. The third is the learned-versus-fixed kernel gap, which decides whether the model or only the protocol is new. The fourth is the degree sweep, which shows how low $d$ can go before recency-sensitive ICU tasks degrade. A clear negative result would also be informative: moment models that trail GRU-D by several points on MIMIC-III even at high $d$ would locate the value of gated recurrence precisely where constant-depth encryption cannot follow.