How Much Personality Leaks Through Federated Adapter Updates?

Phone keyboards and on-device assistants increasingly adapt a language model to each user's own writing through federated learning. In this setup the user's text never leaves the device; only small parameter updates are sent to a coordinating server, which averages them. The usual argument is that this protects privacy. Personality research gives reason to doubt it. Traits such as extraversion, neuroticism and the Myers-Briggs dimensions can be predicted from a person's accumulated posts. An update computed by fitting a model to those same posts may therefore carry the same signal, even though nobody asked the model to learn about personality. We propose to measure this leakage directly. The goal is to turn the vague worry that "updates leak psychological traits" into a number that can be audited, and then to test one cheap, psychologically motivated way of reducing it against the obvious privacy tools.

The setup has four pieces. Each Reddit user in the PANDORA corpus becomes one federated client. The client fine-tunes a low-rank adapter on its own comments using ordinary next-word prediction. A low-rank adapter (LoRA) is a small add-on to a frozen model whose weight change is the product of two thin matrices, $\Delta W = BA$. A curious server then tries to predict each client's personality labels from that client's update alone. The core claim has two parts. First, these updates reveal a substantial, measurable fraction of the personality information a model reading the raw text could recover. Second, most of that fraction lies in a low-dimensional subspace that can be estimated from public data. If so, clients can keep that part of their update on the device, much as psychometric models split personality into a shared population structure and a private individual deviation. The test is whether this protects better than differential privacy at the same language-modeling quality.

What exists and what is missing

Only one paper in the file joins the two directions. Learn to Preserve Personality: Federated Foundation Models in Recommendations proposes splitting a federated model into a frozen backbone, shared adapters $\phi_g$ and private per-user components $\phi_u$. It also proposes a server-side filter that drops client updates carrying "detectable leakage of sensitive psychological traits." It is a position paper. It reports no experiments, leaves the leakage penalty $\Psi(\phi_u)$ undefined, and says that standard differential privacy does not capture semantic trait leakage without testing that claim. Our work supplies what it lacks: an operational leakage metric, a measurement on real personality corpora, and a head-to-head test of whether its filtering idea or differential privacy actually works.

The motivating evidence comes from three places. MineGrad shows that gradient inversion can reconstruct private inputs even from LoRA fine-tuning updates. That makes trait inference plausible, though MineGrad targets text reconstruction, not psychological attributes. PANDORA Talks shows that personality is entangled with demographic predictions. Its gender classifier errs on 14.4% of women versus 8.1% of men, with atypical-trait users misclassified most often. Any defense that treats some personality profiles as "leaky" therefore risks harming exactly those users. The centralized detectors show how much signal the text itself holds. Multi-Document Transformer for Personality Detection treats a user's posts as an unordered set, and TrigNet and PQ-Net add psycholinguistic graphs and questionnaire-guided attention. We use these as the ceiling on what any attacker could learn.

Two further papers supply method. RoLoRA fixes a flaw in averaging adapters. Averaging $A$ and $B$ separately does not average their product, and it restores exact averaging by alternating which matrix is trained each round. With 50 clients on QQP it reports 85.81% accuracy against 70.72% for plain averaging. It is our strongest utility protocol, and its alternation raises a new question we can answer: does sharing only $A$ or only $B$ in a given round leak less? IPGP models personality factor loadings as a population part plus an individual deviation, $\mathbf{W} = \mathbf{W}_{pop} + \mathbf{w}_i$. It beats the population-only model (0.417 vs 0.403 accuracy, Bayes factor $\exp(1.06\times10^4)$). IPGP is centralized and survey-based. We borrow only its structural idea that individual deviation is real signal worth keeping locally rather than discarding.

Core Claim

Define the leakage of a federated protocol as the fraction of text-recoverable personality information that an attacker recovers from updates:

$$L = \frac{F1_{\text{update}} - F1_{\text{chance}}}{F1_{\text{text}} - F1_{\text{chance}}}$$

The three terms are defined as follows. $F1_{\text{update}}$ is the macro-F1 of the best probe we can train that maps a client's shared update to its MBTI dimensions. $F1_{\text{text}}$ is the macro-F1 of a strong detector reading the same client's raw comments. $F1_{\text{chance}}$ is the majority-class macro-F1. For Big Five we use the same form with Pearson correlation in place of F1. $L=0$ means updates reveal nothing beyond chance; $L=1$ means they are as revealing as the text. We hypothesize that $L$ is well above zero for plain federated LoRA. We further hypothesize that projecting each client's adapter update away from a small trait-predictive subspace $U$, estimated on public shadow users, and folding the removed part into a private local adapter, lowers $L$ substantially at little cost in perplexity:

$$\Delta A^{\text{shared}} = (I - UU^\top)\,\Delta A, \qquad \Delta A^{\text{local}} = UU^\top \Delta A$$

Here $U$ has orthonormal columns spanning the directions a public probe relies on. The equation is written for the $A$ matrix; the $B$ matrix is handled symmetrically in RoLoRA's alternate rounds. This is a concrete candidate for the undefined $\Psi$ in the position paper, but it acts by routing the signal locally, not by penalizing it.

Research Program

The first stage is measurement. We simulate cross-device federated learning on PANDORA, whose 10,288 users and 17 million comments give many clients with enough text to fine-tune on. Kaggle MBTI (about 8,600 users with 40 to 50 posts each) serves as a low-data replication. The main sweeps use a decoder model around 1B parameters with LoRA, and we confirm key results on Llama-2-7B with QLoRA, following the setup in From Text to Emoji. Users are split into disjoint groups: shadow users, whose labels the attacker may use to train probes, and target users, on whom leakage is measured.

We test attackers of increasing strength: a linear probe on flattened updates, a small multilayer network, and a probe on the singular vectors of the reconstructed product $BA$. Where feasible, we add a two-step attack that runs MineGrad-style inversion and then feeds the recovered text to Transformer-MD. Leakage is reported per trait, per round, and as a function of how much text a client holds. The per-trait breakdown matters because From Text to Emoji found strongly asymmetric trait expressibility, with LLaMA-2 agreeableness alignment only 0.065. We expect leakage to be similarly uneven.

The second stage compares protocols. We run plain FedAvg-LoRA, RoLoRA's alternating schedule, and FFA-LoRA's frozen-$A$ variant, and ask whether the matrix being shared changes $L$ independently of utility.

The third stage tests defenses on a common frontier. We plot held-out next-word perplexity against leakage for each of the following:

  • the subspace-routing method;
  • client-level differential privacy at several noise levels;
  • the position paper's filter, which discards updates a server probe flags;
  • keeping the whole adapter local;
  • secure aggregation across cohort sizes.

Baselines

The naive reference is plain FedAvg over LoRA matrices with no defense. It sets the leakage we are trying to reduce and the perplexity we are trying to keep. Its utility is known to degrade with many clients; the RoLoRA paper reports 70.72% on QQP at 50 clients. The other lower anchor is a frozen model with no fine-tuning, which leaks nothing and gives the worst perplexity. Together they bracket the frontier.

The strongest utility competitor is RoLoRA. It is the best-performing federated LoRA protocol in the file (85.81% QQP accuracy at 50 clients versus 76.48% for FFA-LoRA). All defenses are applied on top of the RoLoRA schedule, so any perplexity loss is attributable to the defense and not the protocol. No reported result was found on Lacuna for RoLoRA's perplexity or personality leakage on PANDORA, so we measure both ourselves.

The strongest privacy competitor is client-level differential privacy: clipping each client's update and adding Gaussian noise before sending. We sweep the noise level to trace its full perplexity-leakage curve. It receives the same client sampling, number of rounds and hyperparameter search budget as our method. We tune the clipping norm for it separately so it is not handicapped. No reported result was found on Lacuna for differential privacy against personality inference. Our method beats this baseline only if its curve lies below the privacy curve at matched perplexity across a meaningful range, not at a single point.

The strongest ceiling on text-based inference is Transformer-MD, which reports 60.92% macro-F1 on Pandora MBTI and 70.47% on Kaggle MBTI. On Kaggle MBTI, PQ-Net's reported 71.32% is the stronger ceiling. We retrain the relevant detector on our exact user splits to obtain $F1_{\text{text}}$, so that $L$ is computed on identical users.

Three ablations isolate the key ingredient. The first replaces the learned trait subspace $U$ with a random subspace of equal dimension, which tests whether protection comes from targeting trait directions or merely from removing capacity. The second projects without folding the removed part into a local adapter, which tests whether local absorption preserves utility. The third varies the rank of $U$ to find how concentrated the leakage is.

Why not just…?

Why not just use client-level differential privacy? This is the real threat. Differential privacy offers a formal guarantee, whereas our projection offers only an empirical one against the attackers we try. The position paper asserts that differential privacy misses semantic trait leakage but gives no evidence. The frontier comparison settles the question. If differential privacy reaches the same leakage at equal or better perplexity, we report that the simple tool suffices and that trait-targeted routing is unnecessary. That would still be a useful answer for deployers who currently have no leakage number at all.

Why not just use secure aggregation so the server never sees individual updates? Secure aggregation removes the per-client view, but the averaged update of a small cohort can still reflect its members' traits. Deployments also often use cohorts of tens rather than thousands. We measure leakage from cohort averages as cohort size grows. We expect the signal about any one member to fall roughly as one over the square root of cohort size, because independent members add noise that averages down. This tells deployers what cohort size makes the per-client attack moot. If modest cohorts already drive $L$ near zero, that is the recommended defense and our method matters only when secure aggregation is unavailable.

Why not just filter out leaky updates at the server, as the position paper proposes? Discarding updates that a server probe flags as trait-revealing removes those users' data from training. PANDORA Talks shows that trait-atypical users are already the ones models serve worst. We measure per-user perplexity for filtered versus routed clients, broken down by trait profile. We expect filtering to widen the utility gap for atypical users, while routing keeps every client's contribution.

Why not just keep the whole adapter on the device and skip federation? This gives zero leakage by construction. It is a legitimate endpoint, and IPGP's result suggests individual components carry real signal. However, clients with little text overfit. Kaggle MBTI users have only 40 to 50 posts. The full-local baseline therefore shows how much shared learning is worth, and hence how much leakage it is worth tolerating.

Decisive Checks

Three results would settle the proposal. First, if the best probe on plain FedAvg-LoRA updates yields $L$ near zero on PANDORA, the leakage concern is unfounded for this use case and the defense program stops; that negative result is itself worth publishing. Second, if leakage is substantial, the subspace-routing method must beat its random-subspace ablation. This confirms that trait signal is actually concentrated in the targeted directions. Third, it must lie below the differential privacy curve at matched perplexity, and it must do so against an adaptive attacker retrained on already-projected updates. An attacker that recovers most of the leakage after projection would show that the protection was superficial, and we would say so. Throughout, results are reported per trait and per demographic group, so that any privacy gain that comes at the expense of atypical users is visible.