PriTabICL: Differentially Private Tabular In-Context Learning with Task-Aligned Cluster-Wise Synthesis

Syed Ahzaz Shah, Evan Bramblett, Héber H. Arcolezi, Chen Gong, Qianqian Tong, Xiaochen LiarXiv 2026 · arXiv

TL;DR

PriTabICL is a new framework that makes tabular in-context learning (TabICL) private without sacrificing accuracy. It achieves this by partitioning data into differentially private (DP) clusters and generating synthetic demonstrations focused specifically on the relationships between features and the target label.

What Problem Does This Paper Address?

Large Language Models (LLMs) are surprisingly good at "in-context learning"—performing tasks by looking at a few examples (demonstrations) provided in the prompt. For tabular data, this means you can get an LLM to predict things like medical risks or financial outcomes just by showing it a few similar historical records.

The catch? Those historical records often contain sensitive personal information. Simply putting them into a prompt—even if the LLM is "private"—risks leaking data through model responses or membership inference attacks.

Fig. 1. A TabICL example and its privacy concern.

Standard Differential Privacy (DP) techniques usually involve adding noise to the data. However, in high-dimensional tabular data, this noise quickly "drowns out" the subtle correlations between features and labels that the LLM needs to make accurate predictions. Furthermore, global data synthesis often fails to capture patterns in minority subgroups or local data structures.

Methods

The authors propose PriTabICL, a "plug-and-play" framework that works with existing DP synthesis algorithms (like PrivSyn or GEM). It moves away from perturbing individual records and instead builds a reusable, private synthetic "pool" of demonstrations.

The framework operates in three main steps:

  1. DP Clustering (Offline): Instead of modeling the whole dataset at once, PriTabICL uses a private version of K-Means to group similar records. This ensures that the synthesis process can capture local patterns that might be lost in a global average.
  2. Task-Aligned Synthesis (Offline): The authors identify that most attribute-to-attribute relationships are irrelevant for a specific prediction task. They introduce a "feature-target workload" that forces the DP synthesizer to spend its limited "privacy budget" only on the relationships between input features and the target label.
  3. Query Routing (Online): When a new query arrives, it is routed to the nearest DP cluster center. Demonstrations are then sampled from that specific cluster's synthetic pool to ensure they are highly relevant to the query.
Overview of the PriTabICL workflow, showing offline construction and online inference stages.

Datasets

The researchers evaluated PriTabICL on four diverse binary classification datasets, representing different challenges in heterogeneity and dimensionality:

Summary of datasets used in the evaluation.

Key Insights and Ideas

  • Local vs. Global Trade-off: Modeling a global distribution often obscures minority patterns. While clustering splits the privacy budget across multiple groups (increasing noise per group), the benefit of capturing local structure often outweighs the cost of that extra noise.
  • Workload Prioritization is Crucial: In high dimensions, trying to preserve every correlation is a losing game under DP. By focusing strictly on feature-label relationships, PriTabICL significantly reduces the "effective" dimensionality the synthesizer has to handle.
  • Synthetic Data as a Privacy Shield: Once a DP synthetic dataset is released, you can draw infinite demonstrations from it for any number of queries without any further privacy loss. This makes it much more scalable than methods that perturb data on-the-fly for every prompt.

Key Results

  • State-of-the-Art Performance: PriTabICL-PrivSyn outperformed existing private TabICL methods in nearly all 12 evaluated settings (across different datasets and privacy budgets).
  • Minimal Utility Gap: On the "Airline" and "Phishing" datasets, PriTabICL reached or even exceeded the performance of random selection from the original non-private records.
  • Consistency: Across the board, the task-aligned workload provided a consistent boost. For example, on the Adult dataset, it increased the F1 score from 0.676 to 0.728 compared to global synthesis.
  • Efficiency: The offline construction is fast (taking less than a minute for most datasets), and the online inference adds zero privacy-related overhead to the LLM call.

Limitations and Future Work

The authors note that the effectiveness of clustering is dataset-dependent. On datasets that are relatively homogeneous (like the "Adult" dataset), clustering can actually be a slight disadvantage because it fragments the data without providing much "local" insight.

Future directions include:

  • Adaptive Clustering: Automatically determining the optimal number of clusters based on the data's characteristics.
  • Higher-Order Interactions: Extending the workload to capture interactions between multiple features and the label, rather than just pairwise relationships.
  • Advanced Selection: Developing smarter ways to pick demonstrations from the synthetic pool beyond simple random sampling.

Summary updated

Abstract

Tabular in-context learning (TabICL) enables large language models (LLMs) to predict target records using a handful of labeled demonstrations in their prompts. However, these demonstrations often contain sensitive personal information. Differential privacy (DP) offers principled protection, but noise-based methods introduce substantial noise, distorting the feature-label relationships essential for accurate prediction, especially as data dimensionality grows. In this work, we study DP data synthesis for generating privacy-preserving TabICL demonstrations. We identify two limitations of directly applying general-purpose DP synthesis to entire datasets: modeling a global distribution can obscure local structure and minority patterns in heterogeneous data, while measuring broad attribute dependencies under a limited privacy budget can reduce the accuracy available for relationships most relevant to prediction. More broadly, global statistical fidelity may fail to preserve heterogeneous structures needed by downstream predictive and generative models. To address these challenges, we propose PriTabICL, a general framework compatible with existing DP synthesis algorithms. It partitions records into DP clusters and synthesizes each independently, using a fixed feature-target workload to prioritize prediction-relevant statistical relationships. At inference, each query is routed to a DP cluster, and demonstrations are sampled from its synthetic dataset without additional privacy loss for the original records. Experiments on four real-world datasets demonstrate the effectiveness of our methods across heterogeneous datasets. In particular, one of our instantiations, PriTabICL-PrivSyn, improves over global PrivSyn in 8 out of 12 settings and outperforms existing private TabICL approaches in nearly all evaluated settings. Open-source code is available.
BibTeX
@misc{shah2026pritabicl,
  author = {Syed Ahzaz Shah and Evan Bramblett and Héber H. Arcolezi and Chen Gong and Qianqian Tong and Xiaochen Li},
  title = {{PriTabICL: Differentially Private Tabular In-Context Learning with Task-Aligned Cluster-Wise Synthesis}},
  year = {2026},
  eprint = {2611.25073},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.25073},
}
Figure